.webp)
Many applications are now targeting several operating systems. In most cases, the code is similar for each of them and only the toolchain is different (i.e., Visual Studio for Windows, XCode for OS X, gcc for Linux, ...) and the library which permits the portability of the code (i.e., C++-STL, Boost, Qt, ...).
In general, by choosing the right development environment, it is possible to limit the platform code to a few #if/#endif sections. And in most cases, the unit tests and other automatic tests should be of course re-executed on each target platform but it makes no sense to do an intensive manual test cycle on each of them. It is instead more optimal to split the testing effort over all platforms and try to collect and merge the result together.
An other aspect is that it makes sense to find the differences for each platform and execute a minimal set of dedicated tests for each of them. This approach is interesting for embedded systems: if an application can be executed on a host system through some wrapper, it makes sense to run all unit tests on it, then look on what could not be covered and test this part on real hardware.
In all cases, we need to build our application in different environments, collect the result and be able to identify the differences of the code coverage.
Compiling the Same Code on Two Platforms
Let's take a small sample composed of two files:
- One common source which contains only the main() function. This file is common and compiled identically (with the same preprocessor and instrumentation settings) on all platforms.
- One file which contains some #ifdef statement and which executes different code on each platform. In our case, it prints only a different string. So the source code is the same on all platforms, but the preprocessor generates different assembly code, adapted to each platform.
main.c:
extern void print_information();<br>int main()<br>{<br> print_information();<br> return 0;<br>}
information.c:
#include <stdio.h><br>void print_information()<br>{<br>#ifdef UNIX<br> printf( "Unix Systemn" );<br>#else<br> printf( "Other Systemn" );<br>#endif<br>}<br>
The compilation on Unix can be performed with:
$ csgcc -o app_unix.exe information.c main.c -DUNIX
On Windows, it would be:
$ cscl /Feapp_windows.exe information.c main.c
Then on each platform, it is possible to execute the application and import the coverage:
$ ./app_unix.exe<br>$ cmcsexeimport -m app_unix.exe.csmes --title="Test on Unix" app_unix.exe.csexe --delete
And the same on Windows:
$ .app_windows.exe<br>$ cmcsexeimport -m app_windows.exe.csmes --title="Test on Windows" app_windows.exe.csexe --delete
Merging the Coverage of the two Platforms
The coverage on each platform looks as follows:
The coverage is as expected: the main() function is executed and in the print_information() function, the line compiled for this platform ( 'printf ( "Other System\n" );') is also marked as executed.
What we would like to know is if the line compiled for Unix is also covered ('printf ( "Unix System\n" );' ). For that, the first approach would be to merge app_windows.exe.csmes and app_unix.exe.csmes. The result would be:
The result is not really what is expected: we see that main.c and information.c appear twice. If we switch to the tree mode of the source dialog, it is clear why this happens: the source files are not in the same directory.
We then need to tell the CoverageBrowser that these files are identical. For that we use the context menu of the file dialog and click on 'Rename Sources...' and replace the directories to match an identical one with a regular expression:
Actual Name: .*[\\/]([^/\\]*)
New Name: src/\1
The expression patches any string which contains a slash or backslash and extracts the ending parts which do not contain one using the expression ([^/\]*). The parenthesis permit to capture the expression with the placeholder \1. ('1' is for the first parenthesis expression.) We rename the file names by pre-pending src/ to the file name without the path.
The CoverageBrowser then computes the preview of the rename operation. If there are conflicts or errors, the corresponding lines are highlighted in red. It verifies that if two files have the same name after the transformation, they have also the same source code. So it is not possible to try to rename main.c into information.c.
It is permitted that two C++ files have a different pre-processed output but in this case, two versions of the same sources are created. For our sample, this is the case for information.c: the source file is identical, but the pre-processed output is different. That's why we can see two versions, 'information.c #1' and 'information.c #2', of the original source 'information.c'. This also implies that the function print_information() of the file information.c has also two versions.
This file renaming permits to reduce the number of source files to analyze from 4 (2x main.c and 2x information.c) to 3. This seems not to be a big gain, but in most of the real projects which have more than 99% of the source code identical for each platform this reduction permits to get a complexity of code to analyze near to the single platform version.
Scripting the File Renaming
Squish Coco also permits us to script the file renaming with the tool cmedit. cmedit permits first to list the source files with the switch -l:
$ cmedit -l app_merged.exe.csmes
/net/firewall.vpn/export/TMP/sample/information.c
/net/firewall.vpn/export/TMP/sample/main.c
Z:\sample\information.c
Z:\sample\main.c
We get here the same information as with the CoverageBrowser dialog, we can then rename the sources with the switch -r:
$ <code>cmedit --dry-run -r '.*[\/]([^/\]*),src/1,r' app_merged.exe.csmes</code><br> '/net/firewall.vpn/export/TMP/sample/information.c' -> 'src/information.c'<br> '/net/firewall.vpn/export/TMP/sample/main.c' -> 'src/main.c'<br> 'Z:sampleinformation.c' -> 'src/information.c'<br> 'Z:samplemain.c' -> 'src/main.c'
The expressions followed by the -r switch are separated by a comma. The first one is the actual file name to match, the second one is the destination file name and the last one ('r') permits us to specify that a regular expression is used.
--dry-run allows us to execute the command without modifying the file, if the result is as expected, and removing it permits us to commit the changes.
Conclusion
With the source file renaming functionality, it is possible to handle binaries generated on several platforms from the same source. The coverage is mixed so that it is possible to execute a shared test suite only on one platform. Squish Coco checks that the sources are identical before renaming the files, thus allowing us to ensure that the operation is performed on the same product release. But on the other hand, it allows that the pre-processed code is different to handle platform-specific compilations.
