A leading online poker company uses various Squish versions to test web pages and applications using different technologies and on different platforms. We can’t tell you who they are or even show screenshots of their products, but they did allow Graham Abell, one of their QA Automation Engineers, to tell us about their experiences with Squish.
Graham has worked in test automation for around four years. Prior to Squish, he used a variety of test automation tools including WinRunner, QTP, and LoadRunner. We’re grateful for him taking the time to answer our questions so thoughtfully.
Graham’s company develops two applications—a software installer for Windows and Mac OS X, and a client application for Windows and Mac OS X. The installer is downloaded from a web page. Graham’s company tests the web page with Squish/Web and the installer and client applications using Squish/Qt and Squish/Mac—with both applications being tested on both of their target platforms.
At present Graham’s company has 3,000 tests, with plans to expand to 15,000 tests in their next phase of test development.
Why Squish?
Prior to using Squish, Graham’s company only supported Windows for their installer and client application. When they ported from .NET to Qt to broaden their market to include Mac OS X users, they needed to find a new testing tool, since what they used at that time—QTP—did not support Qt or Mac OS X applications. Graham told us:
We reviewed all the tools which supported Qt, and Squish appeared to be the most mature product. Its support for cross platform testing also appealed to us as we had previously been able to test only on Windows.
Graham’s company conducted its search for a suitable tool in 2008. As part of that exercise they contacted a number of companies that provide testing automation products, but froglogic stood out, as Graham points out:
We were in contact with several companies but froglogic was by far the most responsive. This was something which we valued coming from pretty slow customer service with our previous tools.
Graham’s company got an evaluation copy of Squish and developed some proof of concept tests against a very early release of their new client application. Graham also noted that having access to Squish’s source code was a critical benefit since it gave them the flexibility they needed to fully automate the testing of their client application. They were also very happy with froglogic’s licensing model, commenting:
It was refreshing to see a company address licensing in a manner which makes sense for automation. A single developer will need to run tests across various machines and so the Squish licensing model completely matches that real life usage.
Graham also gave three specific technical reasons why his company had chosen Squish over the competition:
- The option to use languages such as Python, which have a very active community and a wealth of modules available makes test development a lot easier—we don’t have to reinvent the wheel at every turn
- Squish’s ability to access the Qt API allows us to interact with the client application and complete tests that wouldn’t otherwise be possible
- Squish’s cross-platform support of multiple technologies is unique amongst test automation tools
In addition, Graham told us that he had found Squish’s support for running tests across multiple machines to be particularly useful.
Squish in Practice
Once Graham’s company started seriously using Squish they were pleased to discover that Squish has a low system overhead compared with the other tools they had used. This meant that they were able to run tests on lower specification machines, and in particular, on virtual machine instances.
Graham’s team makes use of the HP Quality Center™ in conjunction with the Squish integration add-on. They have found this to be particularly helpful in view of the large number of tests they are developing and maintaining.
Apart from accessing Qt’s API, Graham has found the Squish Spy—a tool for identifying application objects and for viewing and verifying their properties—to be the Squish feature that he makes the most use of.
Graham’s team are all experienced with test automation, so when they adopted Squish they didn’t encounter any significant problems learning how to use it— they simply had to translate from what they had done with their old tools to the Squish approach. It wasn’t all plain sailing however, and initially they had some difficulties in accessing aspects of their client application’s state. Once they’d learnt more about Squish’s facilities and with some advice from froglogic, they soon resolved this problem.
One of the main reasons why Graham’s company uses test automation is to keep the amount of manual regression testing to the absolute minimum. They also use Squish, insofar as possible, to support the testing of new features, something done in conjunction with the company’s separate manual QA team.
Graham’s company are unusual in that they don’t make much use of the Squish Object Map, but keep track of their object descriptions externally. Fortunately, Squish’s open design and use of open file formats, means that it is almost always possible to integrate Squish into existing QA practices and procedures.
When switching to Squish, Graham’s team didn’t encounter any significant technical hurdles. However, they did have to rewrite all their old tests as part of the changeover, but they have found the benefits have already outweighed the costs of doing this. And they have now supplemented Squish with some in-house tools developed in Python and .NET to produce the exact testing environment and processes that suit their needs.
Conclusion
For Graham’s company, Squish has made it possible for their QA team to provide better test coverage and improved test reliability. One consequence has been improved application quality. And they’ve also managed to reduce the number of person-hours required for automated testing.
We’ll leave the final words to Graham:
Support from froglogic is unparalleled, I’ve never dealt with a company who consistently respond so quickly to support tickets and always do their best to resolve the issue as quickly as possible.
froglogic’s team would like to thank Graham for taking the time to share his and his team and company’s experience with Squish, and we look forward to a continued successful relationship.
