FlexUnit / ASUnit deathmatch results

• Tweet

• Tweet

As I mentioned a few posts ago, I am looking into how asunit and flexunit can fit into our continuous integration set up. The move into AS3 has given me an opportunity to reassess how the 2 frameworks fit my/our needs. The prior post set out the criteria I was testing against – here is what I found…

Overall

Both frameworks seem to have followed the junit framework, and attempted a port from java to AS3. The good thing about this is that there are very close similarities between usage of the two frameworks, and so switching from one to the other isn’t too much of a ballache. Both setups have the notion of a testRunner and resultPrinter, and I actually found that it was a very simple affair to adapt the code I wrote to use FlexUnit with Cruise Control to work with ASUnit instead.

I did however find a problem when using the ASUnit testListener. When you run a test, the following methods get called on any testListeners you have (in my case its the CruiseControlResultPrinter):

1. startTest
2. If an error / failure occurs:
   • addError
   • addFailure
3. endTest

The problem is that when startTest gets clled, the getCurrentMethod() method of the testCase returns null instead of the testMethod that is about to be executed. If you have had a nosy at my result printer code, you will see that I need this method to return the correct value when startTest gets called so I can append a testCase node with the method name as its ‘name’ attribute. I managed to work around this by allowing a node to be added with a ‘null’ value for its name; then when I try and access the testcase node, it first searches for a node with a matching name attribute, then for a node with a value of null for the name. I didn’t feel very comfortable with this, as it injects a weak link in my code. However, it wasn’t enough for me to strongly favour one framework over the other yet…

The next problem I had with ASUnit was unfortunately for me a show stopper. One of my criteria is that I am able to execute an asynchronous test (as well as asynchronous test setup). Unfortunately I didn’t manage to find a clean solution for doing this in ASUnit. There was a discussion on the mailing list about how one might go about doing this, but unfortunately I couldnt find a solution that worked within my requirements. Luke made a very good point about the difference between asynchronous and event-driven, but nevertheless I still couldnt get an async test to work and correctly report back before the alltestscomplete event gets triggered with the test suite.

One of my other criteria was also that the framework didnt rely on Flex. For me right now, this is secondary as my current work is using Flex. However, it is still important to me to be able to have the ability to build a test suite for a flash based project. As far as I can see, the only real reliance Flexunit has on flex is the testRunner / result printer that ships with the framework. This would mean that if I needed to, I could build a flash only test runner. I think that the name flexunit doesn’t help, and if it were me, I’d be tempted to find a different name.

I also really like the ability to automatically create test suites that ASUnit provides. Johannes pulled out the functionality for this to enable us to do this as a python script, so I guess with different templates, you could get this working for flexunit.

In conclusion

If you have got this far, I guess you will have worked out that I will be going with flexunit for the time being. I’m kind of sad about this – I suppose I’m quite attached to ASUnit, as Ive been using it for a year and a half, and I really like the community spirit behind it. Ultimately though, it doesnt quite do what I need, and I really didnt want to get deeply under the hood of the framework code. I’m going to keep a very open mind on this decision, and see how asunit progresses. If anyone has anything to add to sway my decision one way or the other, I’d love to hear it too…