The feature sets, quality and usability of code coverage products vary significantly.
- Ant integration
Probably most Java projects today are using Ant (or Maven) to manage their build process, including running unit tests (or functional tests).
Thus, Ant integration is one of those features a code coverage tool cannot afford not to have. However, there are subtle differences in how nicely the code coverage related targets fit into an existing build script. Yes, we will see some examples later on. Of course, most tools also provide an alternative, standalone way of running the analysis, either from the command line or via a GUI application.
- Report formats
Another obvious feature a code coverage tool must have is reports. Again, there are differences in the type and quality of supported reports. Some tools provide only textual summaries in Ant console output, others produce huge tables of names and numbers in HTML, and others produce nice pictures, others offer to render all this in PDF as well. We'll see examples of the main types of reports in the next section.
- Source code linking
Somewhat related to the previous item, source code linking is something one can't live without once having gotten the taste of it. In practice, source code linking means that, as a part of the code coverage report, the tool has generated annotated copies of the actual source code, highlighting the parts which are not covered by your tests. I wouldn't be surprised if this particular feature was the single biggest benefactor in code coverage tools reaching critical mass. Seeing the code block that is causing your coverage to stay away from the "green" is a lot more efficient than seeing that a particular method contains some code that isn't covered by the tests. All of our selected examples include source code linking in their feature set.
- Checks
It doesn't take too long after someone has introduced a rule when someone else introduced the idea to enforce that rule. That has also happened to code coverage. Some tools provide a means to pull up the red flag if code coverage drops below a given level. In the context of Ant integration, the build script might typically fail the build until the tests have been reinforced to cover the naked parts of the code.
- Historical reports
Few tools provide a way to collect a history of coverage data and produce historical reports illustrating how your project's code coverage has been fluctuating in the course of time. This is also a nice feature, although some might consider it rather irrelevant.
