In order to try and do this I decided to use as much of the code as possible that the FlexCover guys had written, and only change what I needed to get it working.Once I had it up and running I could worry about making it faster / leaner / more neatly coded…

The approach

So here’s the basic plan:

1. Swap out the default localConnection reporting mechanism for one that will pick and choose depending on environment the app is running in
2. Create a headless agent that will pull the code coverage data collection and reporting side of the coverageViewer into the app
3. Use commandline options to swap between the headless and localConnnection agent
4. Default to using the localConnnection

How it works

When your app compiles using the instrumented SDK created for using FlexCover, your application makes use of a class called CoverageManager. This manager is a patch that allows you to plug in a custom code coverage agent for use in your app. By default it uses a class called LocalConnectionCoverageAgent which broadcasts code coverage metadata to the coverage viewer application. What my patch does is get in and allow you to use a different, headless agent. To do this simply call during the preinitialize event of your main application:

private function injectAgent():void{
CoverageManager.agent = new CoverageAgentSwitch();
}

As a quick test, run your app as you had previously and confirm it still works by sending data over a localConnection to the CoverageViewer. This is expected as by default, the switch will create a LocalConnectionCoverageAgent. In order to use the headless agent you need to set a few commandline properties. First you need to tell the switch that you want to go headless. Then the headless agent needs to know where it’s getting its metadata from, where it should output the code coverage report. The commandline options are:

-coverage-agent-type=headless <em>Headless is only option, anything else will default to localConnectionAgent</em>
-coverage-metadata='/full/path/to/coverage/metadata.cvm'
-coverage-output='/full/path/to/coverage/reportInOriginalFormat.cvr'
-emma-report='/full/path/to/coverage/reportInEmmaFormat.xml'

Note that if you don’t specify the metadata path and at least one of the report formats, the headless agent will log errors but otherwise fail silently.
In terms of the edits I made to get it working, that is pretty much it. The headlessCoverageAgent swc is probably larger than it needs to be and is definitely grossly inefficient. I will update this soon to improve this, but right now I only have time to get this post up.
Obviously you will have to change your build script to pass these new commandline parameters in to the test harness when you run it. If there is any interest, I’ll post my updated build script and test harness modifications that dispense with the log parser altogether and make for a more repeatable build script.

The Result

My build process is now much quicker because I don’t have to wait to be sure that the coverage viewer has initialized. It’s also completely stable and the code coverage trend has been a great motivator for the team.

Code Coverage trend without crazy variance found using the external viewer
Download headless library

Update

A few of you have asked for a copy of the source code so you can play around with it and undoubtedly make it all work better for your needs. Attached here—with no warranty or support—is the headless agent project. In order to get it to work, you will need to have the CoverageAgent and CoverageUtilities projects in your workspace, as well as the modified CoverageViewer project from previous posts. Good luck with it – It’s code I haven’t looked at for a while so don’t hate on me for it :/

The post Code Coverage with Flex – a headless agent for CI builds appeared first on Eyefodder.

So by now I had my code coverage viewer happily spitting out EMMA formatted code coverage reports that my build machine was just dying to consume. All I had to do was get the running of the code coverage test and the outputting of the report to be an integral part of my build. Here is the sequence of events for my build process:

1. clean my output folders (delete and recreate them)
2. Compile the main application using the instrumented SDK. This will generate the .cvr metadatafile that the coverageViewer needs
3. Compile the test harness using the instrumented SDK. This is so that when it runs, the hooks injected by the SDK report coverage data.
4. Launch the code coverage viewer Application (specifying EMMA output file)
5. Run the test harness. When it is finished, call CoverageManager.exit() which will close the viewer
6. Wait for the test results and code coverage report to be available. By implication this means that the test harness and code coverage viewer have quit.
7. Compile the main app using the standard SDK
8. (AIR Apps only) remove test harness and any test data from the output folder
9. (AIR Apps only) package up the application

In theory this is all well and good. In practice there are a few problems:

• The viewer has to launch and parse the code coverage metadata file. There isn’t a simple way to feed this back to the ant script, so you have to get the script to wait. I set mine to 30 seconds, which should be enough. Nevertheless, this sort of thing frankly just irks me – either at some point it will fail, and up until that point, the build is taking longer han it needs to.
• There are two applications being launched by the build process. In order for this to work, you need to launch the coverageViewer with spawn='true' If something goes wrong with the build, the script no longer has control of this process (I might be wrong on this one – correct me if so..)
• The code coverage agent (that gets injected into our test harness by the instrumented SDK) and the viewer communicate via LocalConnection. This allows two flash applications on the same machine to talk to one another. Unfortunately, LocalConnections can be kind of flaky and you have to build a separate local connection for each direction of communication.

The first two I could live with, but when I put this together, I found that there was quite a wide variance of about 10% from build to build of code coverage values:

Code Coverage trend showing wide variance of reporting results

I was left with one of two conclusions:

1. There was a bug/ fragility in the localConnection and it couldn’t be relied on
2. There was a bug in the coverageAgent.exit() code sequence

The way CoverageAgent.exit() works is that you call it from within your application and it:

1. Waits until the application has sent all remianing code coverage data
2. Sends a message to the code coverage viewer asking it to prepare for exit
3. The viewer writes out any report files
4. Exits, and sends a message back to the main application telling it to exit

Somewhere in this back and forth code coverage data was getting lost, and occasionally the build was failing because the report file was not getting written. I could have attempted to debug the code, but in my experience, debugging LocalConnections is a royal ballache. In the end I decided to bring the reporting side of the viewer ‘in-house’ and avoid the need for a second application altogether. My next post will show this and give you a download to the swc so that you can do it yourself…
Just in case you are interested, I have attached the buildfile I used for running the coverageViewer application as part of the build. If you can wait till I post the next update to this, I’d suggest you do
Download build file archive

The post Code Coverage with Flex – ANT build for running the viewer appeared first on Eyefodder.

So as I was starting to look at this, I could see one of three paths:

1. Create a Hudson plugin to consume flexCover’s code coverage report format
2. Add some sort of XSLT transform to my build process
3. Modify FlexCover to be able to output a report format that Hudson understands

Creating a plugin for Hudson certainly seemed like a possibility, but the plugins for EMMA and Cobertura were already there and stable, so it seemed like it would be much simpler to try and create a code coverage report in a format one of these plugins would understand. Creating an XSLT would work for this, but I’m not an XML guroid, so I figured I’d go down the route of the simplest thing that would work for me and hack extend flexcover to be able to output EMMA formatted code coverage reports.
As it turns out this was a pretty simple job, and after a few hours work I managed to create a patch that allows you to output .cvr or EMMA reports with an additional commandline argument. Alex, Joe and I have talked and although my way of doing it works, it is not designed to be scalable or modular to support more formats. They are incorporating the patch, but will probably rework it with these things in mind. They are super busy guys and it wont happen quickly; if you want to in the meantime you can grab the patch from here, but bear in mind it is offered with absolutely no warranties or support

Installation

First thing you need to do is checkout the FlexCover code from Google code (http://flexcover.googlecode.com/svn/trunk/CoverageViewer) and patch it with the file attatched. The patch is created from the project level down. To do this, you can right click the file from within Eclipse and select ‘Team>Apply Patch…’ Follow the instructions and you should be good to go…

Usage

In order to write out the code coverage report file with the viewer, you need to supply the commandline argument:

-output /full/path/to/my/flexcoverreport.xml

This functionality is unchanged, but now you can also specify an EMMA formatted file using the following:

-emma-report /full/path/to/my/EmmaReportName.xml

And that’s about it! My next post covers the challenges I had integrating this into my build process, and how I overcame them…

Download patch file

Update

Sorry this has taken so long to get together, and work out what was going on. I got swamped under a tonne of work, and neglected to update this

The link above is for an updated patch file (and note – I did this patch from the root level of the project), and for your pleasure below is the infamous missing class. I had to hunt through my old email as my old dev machine died a way back and appalingly I didn’t have the file under source control (I think it’s because I was patching someone elses repository). Anyhoo – enough excuses; here is the missing class: now renamed EMMAReportAdapter

The post Code Coverage with Flex – creating EMMA formatted reports appeared first on Eyefodder.

• Its summer
• Work has been insanely busy
• I’ve been beavering away on a new plugin for eclipse

The plugin is designed with one key goal in mind: shortening the develop-test feedback loop. It integrates eclipse and the flexunit framework to make our lives as developers easier.
The plan is to release the plugin on labs as soon as possible (I’m about to go on holiday for a couple of weeks so it’ll be October at the earliest…) but I thought I’d give you an early heads-up on what I’ve been doing when its been too hot to walk the New York streets…

So, here’s what I’ve done so far:

• Modded the flexunit visual layout to work better in an IDE window
• Setup the plugin to switch test runner when you switch between projects
• Modded the flexunit swf to make it easier to re-run the test by clciking on the bar
• Re-run the test when the test harness is recompiled
• Set it up so that the developer can run a subset of tests
• Set it up so that the test harness mxml is templated and can be distinct per project

Plans for the future:

• integrate more tightly with CI build processes
• autogenerate ANT build script
• ability to choose tests on a per-method rather than per-test class basis
• Incorporate other very cool stuff that flexunit has coming down the line

So, for now we are planning to have an internal release; the plugin is definitely alpha and I’d like to keep it to a limited group to iron out bugs, tighten up my bad Java and find out what a cross-section of our developers need. The plan is to release internally to Adobe Consulting, but if you are a forgiving and patient person who could really benefit from this, drop me a note and we can chat…

The post FlexUnit eclipse plugin alpha appeared first on Eyefodder.

So, grab the jar from here and unzip it to your externals/lib directory. Then add the following properties to your build:

<property name="logParser" value="${lib}/FlashLogParser.jar"/>
<property name="flashStatus.location" value="${logs}/status.txt"/>
<property name="flashOutput.location" value="${logs}/TEST-testOutput.xml"/>

Then, change your parseFlashLog target to look like this:

<target name="parseFlashLog" description="parses flash log" depends="clean" >
<java jar="${logParser}" failonerror="true" fork="true">
<arg line="'${flashlog.location}'"/>
<arg line="'${flashStatus.location}'"/>
<arg line="'${flashOutput.location}'"/>
</java>
</target>

Now you should be good to go…
In my next post I am going to do a full dissection of my latest build file – I have spent some time refining it over the last year, and I think its probably helpful to spend some time looking at it bit by bit…

The post Continuous Integration with Flex – a better log parser appeared first on Eyefodder.

So, when I ran my ant build, all went well until the runTest target executed (or at least tried to execute). I got the following build error:

Execute failed: java.io.IOException: debugPlayerMac.app cannot execute

You see the problem is that on a mac, the standalone player (like other applications on the Mac) is actually a folder containing all sorts of cleverness inside. Ant doesn’t know how to execute a folder, so I was a bit stuck. Until, after lots of digging, I found the answer in an old osflash mailing list archive. Basically I have to use the ‘open’ command to launch the app – something like this:

<exec executable="open">
<arg line="${pathToFlashPlayer}"/>
<arg line="${pathToSWFToPlay}"/>
</exec>

I tried this, and it worked nicely. The problem is that my ant build needs to wait until the test harness runs and closes the player before reading the log and parsing the results for cruisecontrol to consume. In the ant script above, the open command only stalls the ant script until it has finished doing its opening magic, so ant gets upset as the test results haven’t been written yet. I slept on this, then couldn’t think of an answer so I sent an email to the ant users mailing list. Mere minutes later I got a reply that helped me work out how to crack this.
Basically, ant has the ability to run two threads, and move on when they are both complete. So in one thread we launch the player, and in another thread we first wait for the file to be available, then we do a check to see if it contains what we want (I am testing against the flag -----------------TESTRUNNEROUTPUTENDS---------------- which I used in my result printer). It worked like a charm:

<target name="runTest" description="runs the test harness" depends="compileTest">
<parallel>
<exec executable="open" spawn="no">
<arg line="${debugPlayer}"	/>
<arg line="'${testHarness.swf}'"/>
</exec>
<sequential>
<waitfor>
<available file="${flashlog.location}"/>
</waitfor>
<waitfor>
<isfileselected file="${flashlog.location}">
<contains text="-----------------TESTRUNNEROUTPUTENDS----------------"/>
</isfileselected>
</waitfor>
</sequential>
</parallel>
</target>

The last step to making this properly cross platform for cruisecontrol was to have the right <exec> command called depending on the OS. Fortunately there is an os attribute you can specify on <exec> and the task will only run if the os matches the platform you are running the task on. A few minutes later, I had my amended runTest target:

<target name="runTest" description="runs the test harness" depends="compileTest">
<parallel>
<exec executable="${debugPlayerWin}" spawn="no" os="Windows XP">
<arg line="'${testHarness.swf}'"/>
</exec>
<exec executable="open" spawn="no" os="Mac OS X">
<arg line="${debugPlayerMac}"	/>
<arg line="'${testHarness.swf}'"/>
</exec>
<sequential>
<waitfor>
<available file="${flashlog.location}"/>
</waitfor>
<waitfor>
<isfileselected file="${flashlog.location}">
<contains text="-----------------TESTRUNNEROUTPUTENDS----------------"/>
</isfileselected>
</waitfor>
</sequential>
</parallel>
</target>

I’m planning to post my completed project build script and step through it in more detail than I did previously. I’ve updated it quite a lot in the last year, and so far, I think its a lot neater. Before I do that, I’m going to post about my removing the python dependency from my continuous integration process.

The post CruiseControl on the Mac – modifying the build script to work x-platform appeared first on Eyefodder.

<components:StateLinkBar dataProvider="{states}"/>

you can download the code here, or just cut and paste from below:

package view.components
{
	import flash.display.DisplayObject;
	import flash.events.MouseEvent;
	import mx.controls.LinkBar;
	import mx.core.UIComponent;
	import mx.events.StateChangeEvent;
	public class StateLinkBar extends LinkBar
	{
		private var measurementHasBeenCalled:Boolean = false;
		private var __pendingStates:Array;
		private var __states:Array;
		public function StateLinkBar()
		{
			super();
			labelField = "name";
		}
		override public function set dataProvider(value:Object):void{
			super.dataProvider = value;
			if(value==parentDocument.states){
				setStateListeners(value as Array);
			}
		}
		override protected function commitProperties():void{
			super.commitProperties();
			if(!measurementHasBeenCalled && __pendingStates){
				measurementHasBeenCalled = true
				setStateListeners(__pendingStates);
			}
		}
		override protected function clickHandler(event:MouseEvent):void{
			if(__states){
				var index:int = getChildIndex(DisplayObject(event.currentTarget));
				var uic:UIComponent = parentDocument as UIComponent;
				uic.currentState = __states[index].name;
				hiliteSelectedNavItem(index);
			}
			super.clickHandler(event);
		}
		private function setStateListeners(arr:Array):void{
			if(!measurementHasBeenCalled){
				__pendingStates = arr;
				invalidateProperties();
				return;
			}
			var p:UIComponent = parentDocument as UIComponent;
			if(p !=null){
				__states = arr;
				//setup listeners
				p.addEventListener(StateChangeEvent.CURRENT_STATE_CHANGE,onParentStateChange);
			}
		}
		private function onParentStateChange(event:StateChangeEvent):void{
			var newState:String = event.newState;
			if(__states){
				var max:Number = __states.length;
				for (var i:Number=0;i<max;++i){
					if(__states[i].name==newState){
						selectedIndex = i
						hiliteSelectedNavItem(i);
					}
				}
			}
		}
	}
}

The post LinkBar tied to component states appeared first on Eyefodder.

So basically, I got started by looking at the mycroft quickstart guide to building your first search plugin. That helped me to understand the basics of what I wanted to do, but really all I needed was a google search restricted to one site, just like typing site:livedocs.macromedia.com/flex/2 [myterm] into google.

The firefox search plugin format is pretty simple. Basically, you tell it what url to use, and then you can append querystring values on to the end of it using the following syntax:

<input name="parameterName" value="parameterValue/>

Then for the user input, you simple use the value ‘user’ instead of writing value=’blah’. So, if you take a look at the google search plugin source (should be found somewhere like ‘C:Program FilesMozilla Firefoxsearchpluginsgoogle.src’) you will see a node that looks like this:

<input name="q" user>

So, what I actually wanted to do is append ‘site:livedocs.macromedia.com/flex/2′ to my ‘q’ querystring value that the user enters. After a bit of digging, I found that you can repeat input nodes of the same ‘name’ and firefox simply joins these values together. so, in order to search just in the livedocs site, my input nodes looked like this:

<input name="q" value="site:livedocs.macromedia.com/flex/2 "/>
<input name="q" user>

You should be able to find my flex plugin source files in a location similar to this:

C:Documents and Settings[USERNAME]Application DataMozillaFirefoxProfilesytb1w5n7.defaultsearchplugins

. If you can’t be arsed to go digging, you can download them here and here

The post How I built those firefox plugins to google search a site appeared first on Eyefodder.

So, the first thing I did was to create a couple of specialised search bar engines using google. My first thought was to try and get google to search two site: locations in the same query, but it just wont let you… as it turned out, the solution I came up with I think is more usable.

The first thing you will want to do is install the search plugins. You can get the flexcoders one here, and the livedocs one here. When you have installed them, restart firefox and your search engines should now appear in the searchbar at the top right of your window

The next thing I wanted to do is combine the functionality of these two together. After a few minutes digging, I found that the advanced search sidebar plugin for firefox does exactly what I need it to do. If you dont have it, you can get it here. When you have the search sidebar installed, open it up and in the ‘within’ dropdown, select ‘Edit Categories…’ Under the category dropdown, click New… create a category called ‘Flex’ and add the two flex search engines.

Hey presto, you can now search across livedocs and flexcoders in one action…

The post Finding Flex Facts Faster appeared first on Eyefodder.

Overall

Both frameworks seem to have followed the junit framework, and attempted a port from java to AS3. The good thing about this is that there are very close similarities between usage of the two frameworks, and so switching from one to the other isn’t too much of a ballache. Both setups have the notion of a testRunner and resultPrinter, and I actually found that it was a very simple affair to adapt the code I wrote to use FlexUnit with Cruise Control to work with ASUnit instead.

I did however find a problem when using the ASUnit testListener. When you run a test, the following methods get called on any testListeners you have (in my case its the CruiseControlResultPrinter):

1. startTest
2. If an error / failure occurs:
   • addError
   • addFailure
3. endTest

The problem is that when startTest gets clled, the getCurrentMethod() method of the testCase returns null instead of the testMethod that is about to be executed. If you have had a nosy at my result printer code, you will see that I need this method to return the correct value when startTest gets called so I can append a testCase node with the method name as its ‘name’ attribute. I managed to work around this by allowing a node to be added with a ‘null’ value for its name; then when I try and access the testcase node, it first searches for a node with a matching name attribute, then for a node with a value of null for the name. I didn’t feel very comfortable with this, as it injects a weak link in my code. However, it wasn’t enough for me to strongly favour one framework over the other yet…

The next problem I had with ASUnit was unfortunately for me a show stopper. One of my criteria is that I am able to execute an asynchronous test (as well as asynchronous test setup). Unfortunately I didn’t manage to find a clean solution for doing this in ASUnit. There was a discussion on the mailing list about how one might go about doing this, but unfortunately I couldnt find a solution that worked within my requirements. Luke made a very good point about the difference between asynchronous and event-driven, but nevertheless I still couldnt get an async test to work and correctly report back before the alltestscomplete event gets triggered with the test suite.

One of my other criteria was also that the framework didnt rely on Flex. For me right now, this is secondary as my current work is using Flex. However, it is still important to me to be able to have the ability to build a test suite for a flash based project. As far as I can see, the only real reliance Flexunit has on flex is the testRunner / result printer that ships with the framework. This would mean that if I needed to, I could build a flash only test runner. I think that the name flexunit doesn’t help, and if it were me, I’d be tempted to find a different name.

I also really like the ability to automatically create test suites that ASUnit provides. Johannes pulled out the functionality for this to enable us to do this as a python script, so I guess with different templates, you could get this working for flexunit.

In conclusion

If you have got this far, I guess you will have worked out that I will be going with flexunit for the time being. I’m kind of sad about this – I suppose I’m quite attached to ASUnit, as Ive been using it for a year and a half, and I really like the community spirit behind it. Ultimately though, it doesnt quite do what I need, and I really didnt want to get deeply under the hood of the framework code. I’m going to keep a very open mind on this decision, and see how asunit progresses. If anyone has anything to add to sway my decision one way or the other, I’d love to hear it too…

The post FlexUnit / ASUnit deathmatch results appeared first on Eyefodder.