At work, we develop an application which makes heavy use of OSGi. And I need to test the bundles we write. Unit tests are not enough here, because the bundles interact a lot with filesystem and database(-s), so we need a plenty of integration/system tests here. I'm still struggling with the problem of how to organize the tests, how to fire them etc. Anyway, one of the things that I'd like to have is information about the code coverage. So, let's try.

Please notice the source code attached at the end of this text.

I've used Cobertura before, so I decided to use it here as well. From the Cobertura FAQ (http://cobertura.sourceforge.net/faq.html) and my previous experiences, I know that it's possible to instrument classes in jar, and for example deploy them on web server, run some test with Selenium or WebTest Canoo and then get the cobertura.ser file which contains the information about code coverage. If it's possible for web applications deployed on web servers why not for OSGi applications ?

the beginnings

So, I've started with the creation of a very simple bundle (hint: there is the source code attached at the end of this text). All it's logic is contained in activator class (which is ugly, but it has to serve as a "proof of concept", nothing more).

The beginning was not very encouraging. I created a bundle, instrumented it with Cobertura, deployed it on Apache Felix, and the result was like:

org.osgi.framework.BundleException: Activator start error in bundle osgi-test-coverage [9].
        at org.apache.felix.framework.Felix._startBundle(Felix.java:1718)
        at org.apache.felix.framework.Felix.startBundle(Felix.java:1588)
        at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:382)
        at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:363)
        at org.apache.felix.shell.impl.StartCommandImpl.execute(StartCommandImpl.java:82)
        at org.apache.felix.shell.impl.Activator$ShellServiceImpl.executeCommand(Activator.java:276)
        at org.apache.felix.shell.tui.Activator$ShellTuiRunnable.run(Activator.java:167)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NoClassDefFoundError: net/sourceforge/cobertura/coveragedata/HasBeenInstrumented
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
        at org.apache.felix.framework.searchpolicy.ContentClassLoader.findClass(ContentClassLoader.java:223)
        at org.apache.felix.framework.searchpolicy.ContentClassLoader.loadClassFromModule(ContentClassLoader.java:94)
        at org.apache.felix.framework.searchpolicy.ContentLoaderImpl.getClass(ContentLoaderImpl.java:166)
        at org.apache.felix.framework.searchpolicy.R4SearchPolicyCore.findClassOrResource(R4SearchPolicyCore.java:471)
        at org.apache.felix.framework.searchpolicy.R4SearchPolicyCore.findClass(R4SearchPolicyCore.java:185)
        at org.apache.felix.framework.searchpolicy.R4SearchPolicy.findClass(R4SearchPolicy.java:45)
        at org.apache.felix.moduleloader.ModuleImpl.getClass(ModuleImpl.java:216)
        at org.apache.felix.framework.Felix.createBundleActivator(Felix.java:3542)
        at org.apache.felix.framework.Felix._startBundle(Felix.java:1666)
        ... 7 more
Caused by: java.lang.ClassNotFoundException: net.sourceforge.cobertura.coveragedata.HasBeenInstrumented
        at org.apache.felix.framework.searchpolicy.R4SearchPolicyCore.findClass(R4SearchPolicyCore.java:198)
        at org.apache.felix.framework.searchpolicy.R4SearchPolicy.findClass(R4SearchPolicy.java:45)
        at org.apache.felix.framework.searchpolicy.ContentClassLoader.loadClass(ContentClassLoader.java:118)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
        ... 18 more
Caused by: java.lang.ClassNotFoundException: net.sourceforge.cobertura.coveragedata.HasBeenInstrumented
        at org.apache.felix.framework.searchpolicy.R4SearchPolicyCore.findClassOrResource(R4SearchPolicyCore.java:486)
        at org.apache.felix.framework.searchpolicy.R4SearchPolicyCore.findClass(R4SearchPolicyCore.java:185)
        ... 22 more
java.lang.NoClassDefFoundError: net/sourceforge/cobertura/coveragedata/HasBeenInstrumented

pom.xml - Cobertura added

Ok, so Cobertura must be available during runtime. All right, here we go. I've added Cobertura dependency to pom.xml:

<dependency>
    <groupId>cobertura</groupId>
    <artifactId>cobertura</artifactId>
    <version>1.9rc1</version>
</dependency>

and updated configuration of the maven-bundle-plugin:

<Import-Package>
net.sourceforge.cobertura.coveragedata.*,
net.sourceforge.cobertura.util.*,
org.osgi.framework,
!*</Import-Package>
<Private-Package></Private-Package>
<Export-Package>
pl.kaczanowscy.tomek.osgi,
net.sourceforge.cobertura.coveragedata.*,
net.sourceforge.cobertura.util.*,
</Export-Package>

Now, everything looked much better - no problems during the deploy or start of the bundle. But, after shutting down Apache Felix, cobertura.ser file was nowhere to be found. :(

explicit creation of cobertura.ser file

So, I checked the Cobertura FAQ, and added these lines to my BundleActivator enforcing the creation of cobertura.ser file during bundle undeployment:

public void stop(BundleContext arg0) throws Exception {
  try {
    String className = "net.sourceforge.cobertura.coveragedata.ProjectData";
    String methodName = "saveGlobalProjectData";
    Class saveClass = Class.forName(className);
    java.lang.reflect.Method saveMethod = 
      saveClass.getDeclaredMethod(methodName, new Class[0]);
    saveMethod.invoke(null, new Object[0]);
    } catch (Throwable t) {
      t.printStackTrace();
    }
}

Now, everything worked almost perfect. After my bundle was started and than stopped cobertura.ser file was generated in felix/bin directory.

The only strange thing was this exception that was printed during the shutdown of Apache Felix (it looks rather silly to me):

-> shutdown
-> Cobertura: Loaded information on 29 classes.
Cobertura: Saved information on 29 classes.
Cobertura: Loaded information on 29 classes.
Exception in thread "Thread-0" java.lang.NoClassDefFoundError: java/lang/Math
        at net.sourceforge.cobertura.coveragedata.LineData.merge(LineData.java:210)
        at net.sourceforge.cobertura.coveragedata.CoverageDataContainer.merge(CoverageDataContainer.java:218)
        at net.sourceforge.cobertura.coveragedata.ClassData.merge(ClassData.java:386)
        at net.sourceforge.cobertura.coveragedata.CoverageDataContainer.merge(CoverageDataContainer.java:218)
        at net.sourceforge.cobertura.coveragedata.CoverageDataContainer.merge(CoverageDataContainer.java:218)
        at net.sourceforge.cobertura.coveragedata.ProjectData.merge(ProjectData.java:142)
        at net.sourceforge.cobertura.coveragedata.ProjectData.saveGlobalProjectData(ProjectData.java:243)
        at net.sourceforge.cobertura.coveragedata.SaveTimer.run(SaveTimer.java:31)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.ClassNotFoundException: java.lang.Math
        at org.apache.felix.framework.searchpolicy.R4SearchPolicyCore.findClass(R4SearchPolicyCore.java:198)
        at org.apache.felix.framework.searchpolicy.R4SearchPolicy.findClass(R4SearchPolicy.java:45)
        at org.apache.felix.framework.searchpolicy.ContentClassLoader.loadClass(ContentClassLoader.java:118)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
        ... 9 more
Caused by: java.lang.ClassNotFoundException: java.lang.Math: cannot resolve package package; (package=net.sourceforge.cobertura.util)
        at org.apache.felix.framework.searchpolicy.R4SearchPolicyCore.findClassOrResource(R4SearchPolicyCore.java:386)
        at org.apache.felix.framework.searchpolicy.R4SearchPolicyCore.findClass(R4SearchPolicyCore.java:185)
        ... 13 more

the coverage report

The coverage report (generated with cobertura-report tool) looks almost ok, except that it shows me all the classes in the bundle (including those imported by me - like net.sourceforge.cobertura.*) and not only my classes from pl.kaczanowscy.tomek.osgi package. This is a little bit annoying, but not a big deal.

step by step

Ok, now if you are curious, I'll tell you exactly how to do this. I assume that you have:

• Apache Felix installed in FELIX_HOME
• Cobertura installed COBERTURA_HOME
• your project root is PROJECT_HOME

I'd encourage you to use the attached source code (but you have to change value of cobertura.dir property in build.xml file).

First, generate your bundle:

cd PROJECT_HOME
PROJECT_HOME> mvn clean package

Now, fire ant build.xml to instrument it.

PROJECT_HOME> ant

Ok, the classes are instrumented and cobertura.ser file was created in PROJECT_HOME. Copy it to FELIX_HOME:

PROJECT_HOME> cp cobertura.ser FELIX_HOME

Time to run the code in OSGi environment.

Run Apache Felix, install bundle, start it and stop it.

FELIX_HOME> java -jar bin/felix.jar

Welcome to Felix.
=================

-> install file:PROJECT_HOME/instrumented/osgi-coverage-cobertura-1.0-SNAPSHOT.jar
Bundle ID: 46
-> start 46
execute method methodExecuted()
-> stop 46
Cobertura: Loaded information on 23 classes.
Cobertura: Saved information on 23 classes.

Ok, the cobertura.ser file has been updated with coverage information, we can generate report now.

FELIX_HOME> cp FELIX_HOME/cobertura.ser COBERTURA_HOME
FELIX_HOME> cd COBERTURA_HOME
COBERTURA_HOME> java -cp cobertura.jar:lib/asm-2.2.1.jar:
lib/log4j-1.2.9.jar:lib/jakarta-oro-2.0.8.jar 
net.sourceforge.cobertura.reporting.Main --datafile cobertura.ser 
--destination report --srcdir PROJECT_HOME/src/main/java

stay tuned - part II will follow

Ok, so I have my "proof of concept". It works, it can be done, I know how to do it. Fine.

...but there are still some things that bother me. I plan to write second part of this text, and try to answer these questions:

1. do I have to change my bundle (add Cobertura dependency) to have it code-coveraged ?
2. how to make report show only my classes, and not every class included bundle ?
3. how to avoid this silly "java.lang.NoClassDefFoundError: java/lang/Math" exception ?
4. will it work on Equinox ? will it work on ServiceMix with Spring-DM configured bundles ?
5. what happens if I have more than one bundle and want to get code coverage for all of them ? Is it possible ?

Ad.1 I think this one cannot be solved - I'd love to test my real (production) bundles, and not some enhanced versions of them, but the way Cobertura works, I don't think I can do it.

Ad.2 I've also tried to instrument only my classes (I've configured ant task to deal with *.class in target folder, and than created jar), but the result was the same - the report showed all the classes, not only mine. Maybe I can somehow configure Cobertura to behave the way I like it, but I don't know how.

Ad.3 I have no idea...

Ad.4 I guess so, but I need to make sure.

Ad.5 I bet it is possible.