In some cases the application you wish to test has many components running on separate nodes in a network, or even on disconnected machines. You can use Clover to test such applications, although some additional configuration is required. This page describes how to configure Clover in order to get a per-test code coverage for distributed business logic.
Please note that having an application deployed on multiple machines does not necessarily mean that the application logic is truly distributed. For instance, an application might run on multiple machines for the sake of load balancing, but do not have communication between nodes. We recommend to read the Using Clover in various environment configurations tutorial and especially to have a look at the "Decision matrix" which can help you to decide which approach would best fit your needs.
When deploying your application in container environments, you should also check to ensure that Clover has sufficient permissions to function.
In case you use Maven, ensure that you're deploying an instrumented version of an application archive. If you use "clover2:setup" then the WAR/EAR is named as usual, but if you use "clover2:instrument" then file has a "-clover" suffix and is usually located in the
On this page:
- Collecting overall coverage from distributed builds
- Collecting per-test coverage from distributed builds
In Clover Distributed Coverage we have two main machine roles:
Clover Server - is a JVM in which Clover sends "test start" and "test end" events in order to inform Clover Clients about test boundaries; Clover opens a port on which it waits for Clover Clients to connect
Clover Client - is a JVM in which Clover will connect to Clover Server
Please note that Clover's Server/Client designation is actually unrelated with your application structure.
Both Clover Server and Clover Client write coverage files to disk.
Clover Server is the place where you will typically execute unit tests (or integration tests). These tests will call application logic on Clover Client #N machines.
If unit tests (or integration tests) are executed by a build script, then the Build Server actually performs a role of the Clover Server.
Report Server is the place where Clover reports are generated - it can be the same physical machine as Build Server, of course.
The diagram above shows a simplified configuration used in the WebApp example. There are only two JVMs used:
- 1st one is a Maven build, which instantiates a container using Cargo Maven Plugin, deploys compiled application and clover.jar, executes unit tests with "-Dclover.server=true" option (Clover records coverage of test classes) and finally creates a report
- 2nd one is a Tomcat container, where instrumented application is running, Clover listens to "test start/end" evens to record per-test coverage (in addition to the global coverage)
Collecting overall coverage from distributed builds
The first step in setting up coverage from distributed builds is to configure Clover for overall coverage reporting.
Step 1: Understanding the Clover 'initstring'
At build time, Clover constructs a registry of your source code, and writes it to a file at the location specified in the Clover initialization string (
initstring). When Clover-instrumented code is executed (e.g. by running a suite of unit tests), Clover looks in the same location for this registry file to initialize itself. Clover then records coverage data and writes coverage recording files next to the registry file during execution. See Clover Database Structure for more information.
Step 2: Choosing a location for the Clover registry
If you are deploying and running your Clover-instrumented code on different machines, you must provide a way for Clover to find the registry file, and provide a place for Clover to write coverage recording files; otherwise no coverage will be recorded.
Clover provides three different ways to achieve this:
- Specify an
initstringthat is a globally accessible file path
initstringshould be an absolute path to the same filesystem location, and be accessible and writable from the build machine and all execution machines. This could be a path on a shared drive or filesystem.
- Specify an
initstringthat is a relative path, resolved at runtime
initstringrepresents a relative path (relative to the current working directory of each execution context). To do this you need to specify
initstringat runtime via system properties
You can override the Clover
initstringat runtime via system properties. Two (three?) system properties are supported:
If not null, the value of this property is treated as an absolute file path to the Clover registry file
If not null (and the
clover.initstringsystyem property is not set), the value of this property is used as the base directory for the file specified at compile-time in the initstring to resolve the full path to the Clover registry.
If not null (and the
clover.initstring.basedirsystem properties are not set), the value of this property is prepended to the string value of compile-time specified initstring to resolve the full path to the Clover registry.
To set one of these properties, you need to pass it on the command line when Java is launched, using the
For application servers, this may involve adding the property to a startup script or batch file.
For methods two and three in the sequence above, you will need to copy the Clover registry file from the location on the build machine to the appropriate directory on each of the execution machines (as part of the test deployment process).
This needs to occur:
a. after the Clover build is complete, and
b. before you run your tests.
Once test execution is complete, you will need to copy the coverage recording files from each remote machine to the
initstringpath on the build machine in order to generate coverage reports.
Step 3: Set your classpath correctly
You must put
clover.jar (or the appropriate Clover plugin jar) in the classpath for any JVM that will load classes that have been instrumented by Clover. How you go about this depends on the nature of the application you are testing and the environment you are deploying to.
In some cases, the
clover.jar must be on the classpath of the actual webserver, not just on the classpath of the webapp that is instrumented. This is to ensure Clover can properly flush its coverage data when the JVM of the webserver is shutdown.
Collecting per-test coverage from distributed builds
The steps below require you to have carried out the previous steps on this page (related to 'Collecting Overall Coverage from Distributed Builds').
Enabling or disabling distributed coverage at runtime
Clover's distributed coverage feature is enabled at runtime by making use of command-line options.
This can be done without the need for re-instrumentation or compilation of source files.
Enabling distributed coverage
Distributed coverage can be enabled via setting this System property:
This will enable distributed coverage with default settings (host=localhost, port=1198, timeout=5000ms, numClients=0, retryPeriod=1000ms, name=clover.tcp.server).
In case when you cannot use default settings, you can pass specific value for any of attributes using the "key=value" syntax passed as clover.distributed.coverage value:
- host - host name of the "Clover Server"
- port - port on which the Clover will listen
- numClients - number of "Clover Clients" to connect until server starts test execution
- timeout - connection timeout in milliseconds
- retryPeriod - inverval between connection retries in milliseconds
- name - name of the Clover server service (URL is host:port/name)
Clover also needs to know which JVM is hosting your unit tests ("Clover Server"), by providing the following system property:
Disabling distributed coverage
Distributed coverage can be disabled by setting this system property on either the Test or the Application JVM:
This will turn off distributed coverage for the JVM in which this is set, regardless of what was instrumented.
Distributed Per-Test Coverage in Clover will now operate when running distributed builds. Detailed reports will now be available.
Server does not wait for clients, despite having numClients != 0 in build configuration
Do not use
-Dclover.distributed.coverage=on runtime option if numClients!=0 was set in instrumentation. The clover.distributed.coverage provided at runtime will override numClients setting from instrumentation, setting it to 0.
As a consequence, your tests on server will start immediately, without waiting for clients to connect. It can result in lower or zero coverage.
Instead of this:
- enable clover.distributed.coverage option in build file or
- use -Dclover.distributed.coverage=numClients=N (where N is a number >= 0) at runtime
Execution of tests hangs when numClients != 0
Server-client dependency loop
It can happen that your "Server" will wait for "Clients" to connect, while Clients will wait until Server starts unit test execution - it depends on how tests are written.
This is a typical case for web applications running in container (like Tomcat, JBoss), when your unit test calls a servlet class (e.g. via HTTP request). The issue is as follows:
- unit tests on <<server>> are waiting until all clients are connected (numClients != 0) but
- none of the clients will connect until a servlet class is loaded in the container, which happens only when the first HTTP request comes (and it will not come, due to the point above)
In order to avoid this circular dependency you have to:
- create a servlet context listener (and instrument it by Clover)
- the class can do virtually nothing
- the listener class will be automatically loaded by container at application deployment (without waiting for any web request)
- as soon as class is loaded by classloader, it will automatically connect Clover recorder instance to the "Clover server"; Clover server will start its execution
Full code example is available on Bitbucket: https://bitbucket.org/atlassian/maven-clover2-plugin (src/it/webapp).