Code Coverage with Flex – a headless agent for CI builds

By Paul Barnes-Hoggett | Published: July 28, 2009

In my last blog post I gave details of how I user the modified code coverage viewer for flex in an automated build to follow the trend of code coverage over time. The trouble with this approach was that there was a problem either with the localConnection in flex or the code that uses it and there was a wide variance of the values being reported. This post shows you how I fixed it by creating a headless code coverage reporter that you can drop into your test harness and remove the need for a second application altogether.

In order to try and do this I decided to use as much of the code as possible that the FlexCover guys had written, and only change what I needed to get it working.Once I had it up and running I could worry about making it faster / leaner / more neatly coded…

The approach

So here’s the basic plan:

Swap out the default localConnection reporting mechanism for one that will pick and choose depending on environment the app is running in
Create a headless agent that will pull the code coverage data collection and reporting side of the coverageViewer into the app
Use commandline options to swap between the headless and localConnnection agent
Default to using the localConnnection

How it works

When your app compiles using the instrumented SDK created for using FlexCover, your application makes use of a class called CoverageManager. This manager is a patch that allows you to plug in a custom code coverage agent for use in your app. By default it uses a class called LocalConnectionCoverageAgent which broadcasts code coverage metadata to the coverage viewer application. What my patch does is get in and allow you to use a different, headless agent. To do this simply call during the preinitialize event of your main application:

private function injectAgent():void{
CoverageManager.agent = new CoverageAgentSwitch();
}

As a quick test, run your app as you had previously and confirm it still works by sending data over a localConnection to the CoverageViewer. This is expected as by default, the switch will create a LocalConnectionCoverageAgent. In order to use the headless agent you need to set a few commandline properties. First you need to tell the switch that you want to go headless. Then the headless agent needs to know where it’s getting its metadata from, where it should output the code coverage report. The commandline options are:

-coverage-agent-type=headless <em>Headless is only option, anything else will default to localConnectionAgent</em>
-coverage-metadata='/full/path/to/coverage/metadata.cvm'
-coverage-output='/full/path/to/coverage/reportInOriginalFormat.cvr'
-emma-report='/full/path/to/coverage/reportInEmmaFormat.xml'

Note that if you don’t specify the metadata path and at least one of the report formats, the headless agent will log errors but otherwise fail silently.
In terms of the edits I made to get it working, that is pretty much it. The headlessCoverageAgent swc is probably larger than it needs to be and is definitely grossly inefficient. I will update this soon to improve this, but right now I only have time to get this post up.
Obviously you will have to change your build script to pass these new commandline parameters in to the test harness when you run it. If there is any interest, I’ll post my updated build script and test harness modifications that dispense with the log parser altogether and make for a more repeatable build script.

The Result

My build process is now much quicker because I don’t have to wait to be sure that the coverage viewer has initialized. It’s also completely stable and the code coverage trend has been a great motivator for the team.
better coverage
Code Coverage trend without crazy variance found using the external viewer
Download headless library

Update

A few of you have asked for a copy of the source code so you can play around with it and undoubtedly make it all work better for your needs. Attached here—with no warranty or support—is the headless agent project. In order to get it to work, you will need to have the CoverageAgent and CoverageUtilities projects in your workspace, as well as the modified CoverageViewer project from previous posts. Good luck with it – It’s code I haven’t looked at for a while so don’t hate on me for it :/