As mentioned in a previous post, we have been working on a new official release of the Qt Visual Studio Tools. Ahead of this release, we would like to present some key changes and what impact they will have in usability and performance. In this post, we describe a new approach to the integration of the Qt tools moc, rcc and uic, both in the build process and at design time. In a follow-up post we will then discuss how these and other modifications contribute to an improvement in performance.
Custom Build Tools
Integrating external/third-party tools in Visual Studio has traditionally been accomplished by means of "Custom Build" steps associated to files in the project. The files could be, for example, headers containing Qt macros which must be processed by moc to generate the corresponding meta-object code. In this case, each header file is labeled as a custom build item and will have a custom build property page:
This approach has some disadvantages:
"In older versions of Visual C++, you can specify, for a specific file, a custom command that gets executed before the C++ compilation step. This functionality, called Custom Build Steps, has some limitations when working with large projects and complex external tools. You cannot set up a custom build command to be run on all files of a particular type. Also, just for using an external application as a custom build step, you need to have extended knowledge of the configuration flags available on the command line."
https://msdn.microsoft.com/en-us/library/aa730877(v=vs.80).aspx
Unlike the tools shipped with Visual Studio, such as the compiler or the linker, which have their own property pages, custom build steps are limited to a single text field where the path and arguments to the external tool must be specified. Also, as noted, a custom build step cannot be associated to a group of files – such as: "all headers containing Qt macros" – even though it would be possible to have only a single, global set of properties at project level. Each file must have its own custom build step information, which will be stored independently in the project file, regardless of repetition. This can lead to performance issues with large projects that make extensive use of custom build tools, as is the case with Qt projects.
MSBuild and Visual C++
In more recent versions of Visual Studio, MSBuild has become the underlying build engine for Visual C++. The content of C++ project files (*.vcxproj) is formatted according to the MSBuild XML schema and the build process is organized into MSBuild targets and tasks. Built-in tools, like the compiler and the linker, are parameterized through definitions – called "Rules" in Visual Studio-speak – which include the set of properties associated to a particular item type (e.g. C++ source files). The syntax of rule definitions also follows the MSBuild XML schema.
As an alternative to custom build steps, an external tool can be integrated into the build process by providing specialized MSBuild rules and targets. Instead of having to rely on the pre-defined behavior of the custom build mechanism, with this alternative approach it is possible to control the way the external tool is invoked using the full expressiveness of MSBuild. Also, by defining a rule for the external tool, project files can then be labeled as inputs to that tool rather than as generic custom build items. Instead of providing just one text field for the command line, Visual Studio will be able to provide those files with a property page specific to the external tool.
Qt and MSBuild
The new release of the Qt Visual Studio Tools will include MSBuild rules and targets specific to moc, rcc and uic (Qt/MSBuild for short). It will be possible, for example, to label header files as moc inputs. The property page that Visual Studio will display for these files will be specific to moc, and will contain fields covering all the options of that tool (the same will also apply for rcc and uic). As with internal Visual Studio tools like the C++ compiler, properties can be defined either for specific files or globally at project level:
Projects created with the new version of the Visual Studio Tools will use Qt/MSBuild instead of custom build tools, and it will be possible to convert custom build tools in older projects. This conversion is not mandatory; the new version of the VS Tools will still work with custom build tools as before. When importing .pro files, the conversion of custom build tools to Qt/MSBuild will automatically take place.
The XML files with the Qt-specific rules and targets will be included in the VS Tools installation package and will be available to anyone that installs it. There is, however, no dependency to the VS Tools. Sharing a project that uses Qt/MSBuild (for example, with someone that does not have the VS Tools installed) will just require that the corresponding XML files be copied together with the project files.
The full expressiveness of MSBuild provides access to new possibilities which were not available with custom build tools. For instance, it is now possible to run moc, rcc and uic in parallel processes instead of having to go through every file in sequence, which was the only option with custom build tools. This feature is implemented in Qt/MSBuild using two targets. The first target goes through all input files, generates the command line to run for each file and adds it to a list of work to do. Another target then reads from this list and spawns the corresponding processes. The result, as expected, is a visible decrease in processing time:
Another important benefit of using MSBuild is the possibility to directly influence the build process, including what project items are processed and with what tools. Inside an MSBuild target, it's possible to add new items to the build and choose with which tools to process them. The Qt/MSBuild targets make use of this feature, for instance, to handle C++ source files containing Qt class definitions. These files are manually added to the project labeled as moc inputs, and are then dynamically reintroduced into the build process as inputs to the C++ compiler.
Furthermore, by using the dynamic source feature of MSBuild, source files generated by the Qt tools can be added "on-the-fly" to the C++ compilation during the build process. As such, there will be no need to include generated files in the project. This has a positive effect on the performance of the VS Tools, which will be part of the discussion in the next blog post.
Future Improvements
The new approach to the integration of moc, rcc and uic may potentially break some use cases where a file needs to be processed by a Qt tool and by some other third-party tool. This type of use case is currently possible given that the custom build is a generic mechanism that can be used to invoke any number of commands. Taking the example of a header file containing Qt macros, there is no guarantee that only moc will be called in that file's custom build step. The conversion of custom build to Qt/MSBuild will only consider calls to moc, rcc and uic; all other commands in the custom build steps will be ignored.
One such use case concerns the integration of the tool IncrediBuild. If this tool is installed, when importing a .pro file using the current version of the VS Tools the custom build steps will contain additional instructions for IncrediBuild to run moc, rcc and uic in parallel/distributed processes. For the upcoming release, this integration will no longer be supported. The parallel execution of Qt tools will still work when using IncrediBuild but only on the local machine (built-in tools like the C++ compiler will still be able to run on distributed processes). A specific fix for the integration with IncrediBuild will be available in a later version of the VS Tools. As a generic solution to this issue of additional commands to run together with the Qt tools, we could consider allowing those commands to be given as pre- or post-events in the moc, rcc and uic property pages.
Wrap-Up
We have presented the outline of a new approach to the way moc, rcc and uic are integrated in Visual Studio. This new approach provides a design time experience and build time integration identical to that of built-in tools. In a follow-up post, we will discuss what impact these and other changes to the VS Tools will have in terms of performance. Hopefully, users will find that, with these changes, developing with Qt on Visual Studio will become even more productive.
