Squish - Test automation tool for our HMI build with Qt

When test engineers hear about test automation the first word that comes to mind is of course Selenium which is the most popular testing library that helps us writing scripts for web applications. There are also ready solutions for mobile apps like Appium, Robotium, Espresso, UI Automator and others. The challenge is when we have some project-specific technologies that are not as easy to automate as web applications. But while using Qt we have some advantage over other non-web applications because there is some ready solution that we can use. 

Graphics1

Our Addrive project – digital cockpit application

The goal of the project was to build a digital cockpit system for car sharing solutions with navigation as the main feature, where the user agrees on advertisements while choosing a cheaper subscription. Advertising can suggest purchasing coffee to the driver, which can be ordered through our application from the navigation screen, then we add a coffee stop to the destination point. We also included the HVAC module and menu where the driver can switch between different screens (music player, settings, 3D model of a car, phone, weather view). To build this we were using Qt Application Manager, QML, MapBox, Qt 3D Studio and server written in python that was using OSRM.

As you can see our application is well-designed so we wanted to make sure that all the images are in place, ads are displaying correctly, we can switch between different screens, increase and decrease temperature and fan levels and other relevant features we had.

Test automation options

After doing research regarding the test automation tools for Qt we find out that there are three options: writing own testing framework, kd executor and Squish for Qt.

The first option was rejected immediately because we had very little time before Qt World Summit on which we wanted to present our demo application. KD executor is basically some simple macro tool that records exact mouse/click steps then the test should be run on the same environment which may result in plenty of flaky tests and it is impossible to run test scripts on the different environment not to mention running this on an embedded device. And then comes Squish which seemed the best solution in our case.

Squish injects into Qt application so one can manipulate with object and models like it is done on HTML DOM with Selenium. In addition, it supports all Qt widgets, QML and Qt Quick controls, cross-platform and cross-device cases, BDD approach and can be run on Continuous Integration tools. And it’s entry-level is very low! Looked like a perfect tool for our project.

Setting up Squish

To install this tool you need to register for an evaluation license here (note from me: people working in froglogic provides very good support, don’t hesitate to contact them if you run into any troubles). After registering you should be able to download binary files (or .run for Linux systems), keep in mind that you need Squish compiled against the same version of Qt as your project is. After short installation process and after running squish some of you will immediately notice that the screen is very similar to Eclipse and that’s true, it is some modification of this IDE with additional features added.

Press File -> New Test Suite and you will see it’s creation screen. Browse for the application that you want to run and if additional arguments are needed you can provide them later on the AUT (Application Under Test) subpage.

Graphics 2

In AUT subpage you can specify more arguments for your application

Recording the test

After we have our Test Suite setup ready and AUT is specified we can start creating automated scripts for our test cases. Now it's time to create a new file in our suite after that the easiest way to create a test is to start recording it by simply clicking "record" on top bar of our IDE.

Graphics 3

Recording the test

 

If we configured everything correctly our application will start so we are able to interact with it as standard users do. You should also see a new window - Control Bar.

Graphics 4

Test recording control bar

Property verification

Try to perform only the steps that are described in your test case and if you need to insert any assert/verification point then just expand the menu near the "Verify" button. Here we have plenty of options, we can either check for different properties, make a screenshot to compare while test will be ran, check the table or make some more precise visual verification or often we can combine that verification into one point. After choosing any of that option we need to choose the object that we want to verify. We can do it either by expanding the object tree and searching for the element that is interesting for us or using the pick tool which is represented by the probe icon. After that, we can check multiple properties to verify the selected object for ex. height, width, source of it, their parents and plenty of others. It all depends on how the application is designed and built.

Graphics 5

Picker and property list

Visual verification

Similar steps are needed for screenshot/visual verifications, after picking the area or element that we want to verify we need to select the object and then the screenshot is made. With Visual verification option, we have three additional options that can be verified: content - so we can also check properties, geometry, and screenshot.

Graphics 6

Visual verification point window

When our visual verification point is saved we can specify multiple additional options, for example in how much correlation percentage our image should be exact or we can set positive and negative areas on our image that should or should not be checked. 

After we are done with recording our steps we can use control bar and finish recording the test, all code should be generated in the previously created file. Now is the time to clear our code with unnecessary lines that were recorded and refactor it a bit just to make it more readable and easier to maintain in the future. Take a look at some simple test case that is manipulating our HVAC temperature module:

Graphics 7

Test script

Summary

As you could see Squish is a tool that is simple to start automation with and a perfect pick for the projects where we need to verify the GUI of our Qt application especially when we don't have time in a project to build our own framework. We can basically start right away with creating tests and they can be run from command line interface which means simple integration with continuous integration that we already have in the project and it has support for BDD if needed! So if you would like to start using Squish you might find froglogic's tutorials or knowledge base as very useful places to get some more information. 


Blog Topics:

Comments