Java code coverage tools


Java code coverage tools are of two types: first, tools that add statements to the Java source code and require its recompilation. Second, tools that instrument the bytecode, either before or during execution. The goal is to find out which parts of the code are tested by registering the lines of code executed when running a test.

JCov

JCov is the tool which has been developed and used with Sun JDK from the very beginning of Java: from the version 1.1. JCov is capable of measuring and reporting Java code coverage. JCov is distributed under the terms of the GNU Public License. JCov has become open-source as a part of OpenJDK code tools project in 2014.

Features

JCov is capable of reporting the following types of code coverage:
JCov implements two different ways to save the collected data:
JCov works by instrumenting Java bytecode using two different approaches:
JCov has a few more distinctive features which include, but are not limited to:
JaCoCo is an open-source toolkit for measuring and reporting Java code coverage. JaCoCo is distributed under the terms of the Eclipse Public License. It was developed as a replacement for EMMA, under the umbrella of the EclEmma plug-in for Eclipse.

Features

JaCoCo offers instructions, line and branch coverage.
In contrast to Atlassian Clover and OpenClover, which require instrumenting the source code, JaCoCo can instrument Java bytecode using two different approaches:
And can be configured to store the collected data in a file, or send it via TCP. Files from multiple runs or code parts can be merged easily. Unlike Cobertura and EMMA it fully supports Java 7, Java 8, Java 9, Java 10, Java 11, Java 12, Java 13 and Java 14.

Tools using or including JaCoCo

OpenClover is a free and open-source successor of Atlassian Clover, created as a fork from the Clover code base published by Atlassian in 2017. It contains all features of the original Clover. The OpenClover project is led by developers who maintained Clover in years 2012–2017.
OpenClover uses source code instrumentation technique and handles Java, Groovy and AspectJ languages. Some of its features include: fine control over scope of coverage measurement, test optimisation and sophisticated reports.
OpenClover integrates with Ant, Maven, Gradle, Grails, Eclipse, IntelliJ IDEA, Bamboo, Jenkins, Hudson, Griffon, SonarQube and AspectJ.

Cobertura

Cobertura is an open-source tool for measuring code coverage. It does so by instrumenting the byte code.

Serenity

Serenity is an open-source toolkit for measuring and reporting Java code coverage. As well as coverage, major code metrics are measured:- cyclometric complexity, stability, abstractness and distance from main. The report data is persisted to an object database, and made available via Jenkins/Hudson. The interface replicates the Eclipse IDE interface visually.
Serenity dynamically enhances the byte code, making a post-compile step unnecessary. Ant and Maven projects are supported. Configuration is done in xml, an Ant example would be:





And a Maven configuration example would be:


-Dincluded.packages=your.package.name.here
-Dincluded.adapters=coverage,complexity,dependency

-javaagent:serenity/serenity.jar -Xms512m -Xmx1024m $ $

For a full example of a configuration please refer to the Jenkins wiki at https://wiki.jenkins-ci.org/display/JENKINS/Serenity+Plugin.
Jenkins slaves as well as Maven multi module projects are supported.

Testwell CTC++ for Java

Testwell CTC++ is a code coverage tool for C, C++, Java and C#. The development of this tool started in 1989 at Testwell in Finland. Since 2013 support and development has been continued by Verifysoft Technology, a company from Offenburg, Germany. Testwell CTC++ analyses for all code coverage levels up to Modified condition/decision coverage and Multicondition Coverage. The tool works with all compilers.

IntelliJ IDEA Code Coverage Agent

IntelliJ IDEA Code Coverage Agent is a code coverage tool integrated in IntelliJ IDEA IDE and TeamCity CI server. It supports branch coverage and per-test coverage tracking.

Notable historic tools

Clover

Clover is a Java code coverage analysis utility bought and further developed by Atlassian. In April 2017 Atlassian announced end-of-life of Clover and at the same time open-sourced it under Apache 2.0 license.
Clover uses a source code instrumentation technique, which has its advantages and disadvantages. Some of its features include historical reporting, huge control over the coverage gathering process, command line toolset and API for legacy integration and more.
Clover also allows testing time to be reduced by only running the tests that cover the application code that was modified since the previous build. This is called Test Optimization and can lead to huge drops in the amount of time spent waiting for automated tests to complete.
Clover comes with a number of integrations both developed by Atlassian and by open source community.
In April 2017, Atlassian announced that they would no longer release new versions of Clover after version 4.1.2, and its code was made available as open-source software hosted on Bitbucket.

EMMA

EMMA is an open-source toolkit for measuring and reporting Java code coverage. EMMA is distributed under the terms of Common Public License v1.0.
EMMA is not currently under active development; the last stable release took place in mid-2005. As replacement, JaCoCo was developed.
EMMA works by wrapping each line of code and each condition with a flag, which is set when that line is executed.

Features