Andrew Ferrier

I spent last Thursday and Friday in London at the Google offices in Victoria for the first Google Test Automation Conference. The presentation topics ranged widely, considering the relatively narrow scope of the conference, but most were well developed and interesting, even if some retrod familiar topics. Some of the highlights included:

• Steve Loughran and Julio Guijarro, HP Labs. This presentation was about Smartfrog, a system deployment framework, which Steve and Julio were working on as part of a strategy for system testing. They demonstrated several examples of how the system might work in practice. Smartfrog looks pretty flexible, and I plan to spend some time looking into it. Frameworks for deployment have an inherent problem in catering to the wide variety of platforms, configuration mechanisms, deployment combinations and so on that are necessary in practice. Anything that gets closer to this is therefore welcome. Smartfrog also has the interesting property that the XHTML it produces as output is sufficiently well-formed that, although it has an embedded CSS stylesheet for presentation in a web browser, it can also be parsed as XML data without much effort, and thus act as a machine-readable data source as well. This might seem obvious to some folks, and I’m willing to bet it’s not the first time it’s been done, but it seemed novel to me.
• Robert Chatley (a fellow alumnus from Imperial College) and Tom White, Kizoom. Tom and Robert were talking about what they called literate functional testing. Essentially this involves creating tests, in this case written in Java, that use plain English for method names, variables, and so on. This means that once punctuation is stripped out, Java code – test assertions – become statements that are readily understandable by non-programmers (such as the business analysts in their organisation). Their framework will shortly be available on Google Code.
• Andreas Leitner, ETH Zurich: Andreas discussed automated testing using contracts. Essentially this means using a language which has pre-conditions and post-conditions on methods, and optionally assertions on objects. He has developed a testing framework called AutoTest, using the Eiffel language, which has these features built in (similar extensions are available for more mainstream languages such as Java). Once these restrictions are placed on a program, generated input can be used to determine whether methods behave as they should. A number of strategies are available for generating this data, and they are pluggable into Andreas’s framework. The simplest is of course to generate the data randomly, but other, more sophisticated strategies are available to improve coverage. Andreas stressed that this type of testing, which is essentially fuzz testing with intelligence, should be used to complement human-created unit tests, not to replace them.
• Goranka Bjedov, Google: Goranka explained some of the background to performance testing, including the differences between performance testing, stress testing, load testing, scalability testing, etc., and how to deploy and manage performance testing systems.
• The conference finished with 10 lightning talks. Subjects covered included jMock (a mock objects framework that complements JUnit), justifying automated tesing in financial terms, testing heresies, Yandex (the largest search engine in Russia, their share ~60%, Google’s ~6%), ‘Automated Testing: Why bother?’, automated tricks for manual testing, and the Perl-inspired Test Anything Protocol. I hadn’t seen lightning talks before, and I thought they were a fantastic idea – similar in some ways to the Straight 8 showings I wrote about a while ago – if you don’t like what you’re seeing, something else is coming along soon. More presentations should be like this.

My thanks to the guys at Google for hosting this conference, particularly for free. The original call for attendees was on their research blog. Some of the conference topics were quite academic and in-depth, but that provided a good constrast to the more practical topics. Google’s offices and facilities are also impressive – definitely worth visiting if you get the chance.

Google LTAC