Code Coverage Integration

Code coverage metrics is the next tool that I want to make available in my project. Poor coverage metrics are an easy way to look for potential bugs. Alas! The opposite is not true, even with 100% line coverage, and apparently even with 100% branch coverage, one can at best expect to find around 60% of the defects in the code. Or at least, that is what the relatively small amount of literature on the subject seems to suggest. Don’t take my word for it, go buy this book, and read the sections about testing. Before you organize a mob and start passing the pitchforks and torches, remember that this does not mean that unit testing (or automated testing) is useless. It is a relatively cheap way to filter many errors, just not an infalible one.

Build Matrix

At this point the number of builds is getting complicated. We want to build with clang and gcc, we want to get code coverage data, and we want to generate the doxygen documentation. Before the .travis.yml file gets out of hand we need to do some basic refactoring and take advantage of the environment matrix. As usual, that was much harder than anticipated, but now I have a build for clang dbg (without any optimizations), which also uploads the doxygen documentation; a build for gcc with code coverage, which also uploads to coveralls.io; and builds for both gcc and clang with all optimizations.

I added the coveralls.io badge, though the state is shameful at the moment. I have been trying to get lcov and llvm-cov to cooperate without success, I am interested because clang promises to deliver branch coverage, a far more interesting metric than line coverage in my opinion. But this is a nice to have more than a required feature.

Ubuntu 12.04 is irritating me.

My experience with lcov is always more frustrating than it needs to be. Needless to say, the default version of gcov crashed miserably with code generated by gcc-4.9 on Ubuntu 12.04. It seemed reasonable enough to assume that the stock lcov with the right --gcov-tool option would work. Nope, no luck. So we need to build a more recent version of lcov in addition to installing more recent versions of gcov and gcc.

Lucky for me, somebody else had solved this problem before.