Qt Quick on Vulkan, Metal, and Direct3D - Part 3
In part 3 of our series on Qt graphics (part 1, part 2), we will look at how shaders are handled in Qt Quick in Qt 5.14 when switching the scenegraph over to rendering through QRhi, the Qt Rendering Hardware Interface. We choose to cover shader handling before digging into the RHI itself because Qt Quick applications using ShaderEffect items or custom materials have to provide fragment and/or vertex shader code themselves, and therefore they need to be aware of (and by Qt 6, migrate to) the new approach to shader handling.
Speaking of Qt 6: while everything described here applies to, and only to, Qt 5.14, and may change in later releases, what we have here will likely form the foundation of graphics and compute shader handling in Qt 6, once the few remaining rough edges are eliminated.
Why something new?
Looking at the qtdeclarative source tree (i.e. the git repo containing QtQml, QtQuick, and related modules), and digging down to the shaders directory with the vertex and fragment shaders for the Qt Quick scenegraph's built-in materials, reveals that Qt Quick already ships with two versions of each GLSL vertex or fragment shader:
Why? This is due to supporting core profile OpenGL contexts (for version 3.2 and up). As OpenGL implementations are not required to support compiling GLSL 100/110/120 shaders in such a context, Qt has no choice but to ship with two variants: one suitable for OpenGL ES 2.0, OpenGL 2.1, and compatibility profiles, and another one (version 150 in practice) that is used only when the context turns out to be a core profile one. As described in part 1 of the blog series, this is essential in order to allow application developers to decide what OpenGL context to request when their application combines its own custom OpenGL rendering and a Qt Quick based UI; regardless of the context being a compatibility or core profile context, Qt Quick will still be able to render.
This is sort of okay when the number of variants is 2. What if now we need Vulkan-compatible GLSL, HLSL, and MSL in addition? Well, the approach does not really scale, unfortunately.
Some of the newer graphics APIs have no built-in shader compilation support anymore, unlike OpenGL. (good bye glCompileShader) And even if they do, as a separate library at least, they may not offer reflection capabilities at run time, meaning there is no way to dynamically discover what vertex inputs and other shader resources a vertex, fragment or compute shader expects, and what the layout of these resources is. (for example, what are the names and offsets of the members in a uniform block)
An internal detail: the Qt Quick scenegraph's batching system relies on rewriting the vertex shader slightly for materials that are used with a so-called merged batch (which is what we get when multiple geometry nodes end up generating one single draw call). Rewriting the shader on the fly before passing it to glCompileShader is suitable when there is only a single shading language in use, but does not scale when we have to implement the same logic for multiple different languages.
- glslang, a compiler from (OpenGL or Vulkan style) GLSL to SPIR-V, an intermediate representation.
- SPIRV-Cross, a library for doing reflection on SPIR-V and disassembling it to high-level languages, such as, GLSL, HLSL and MSL.
So if we "standardize" on a single language, like Vulkan-style GLSL, compile it to SPIR-V, we have something suitable for Vulkan. If we then run the SPIR-V binary through SPIRV-Cross, we get the reflection information we need, and can generate source code for various GLSL versions, HLSL, and Metal Shading Language.
(yes, GLSL is essential still because while there are extensions for OpenGL to consume SPIR-V, counting on that is just not going to work in practice as such an extension will likely be missing on 90% of Qt's target platforms and devices - for instance, OpenGL ES 2.0 is still a thing, even in 2019)
Finally, pack this all together (including the reflection metadata) into an easily (de)serializable package, and there we have our solution.
Therefore, the pipeline that is utilized when running a Qt Quick application with QSG_RHI=1 set is currently:
[ -> generate batching-friendly variant for vertex shaders]
-> glslang : SPIR-V bytecode
-> SPIRV-Cross : reflection metadata + GLSL/HLSL/MSL source
-> pack it all together and serialize to a .qsb file
As of Qt 5.14 this in practice means looking for
The .qsb extension comes from the name of the command-line tool performing the steps above - qsb, short for Qt Shader Baker. (not to be confused with qbs)
At run time the .qsb files get deserialized into QShader instances. This is a fairly simple container, following standard Qt patterns like implicit sharing, and hosting multiple variants of source and bytecode for one shader, together with a QShaderDescription which, unsurprisingly, contains the reflection data. Like the rest of the RHI, these classes are private API for the time being.
The graphics layer consumes QShader instances directly. Graphics pipeline state objects specify a QShader for each active shader stage. The QRhi backends then pick the appropriate shader variant from the QShader package.
- SPIR-V 1.0 when targeting Vulkan,
- HLSL source or DXBC for Shader Model 5.0 when targeting D3D11,
- Metal 1.2 compatible MSL source or pre-compiled metallib when targeting Metal,
- GLSL source for one of version 320 es, 310 es, 300 es, 100 es when targeting an OpenGL ES context (in that priority order, starting from the highest version the context supports),
- GLSL source for one of version 460, 450, ..., 330, 150 when targeting an OpenGL core profile context (in that priority order, starting from the highest version the context supports), and
- GLSL source for one of version 120, 110 when targeting a non-core OpenGL context (in that priority order).
The HLSL and MSL entries in the above list may look a bit strange at first. That is because while we can compile HLSL and MSL from source at run time (our default approach), some experiments have already been made to allow including pre-compiled intermediate formats in the .qsb packages. In practice this means invoking fxc (no dxc support yet - it is in the plans as well but will only be really relevant once we start looking at D3D12) or the Metal command-line tools before the "pack it all together" step in the pipeline shown above. The challenge here is of course that such tools are tied to their platform (Windows and macOS, respectively) so qsb can only invoke them when running on that platform. So when manually generating a .qsb file on Linux, for example, this is not an option. This will likely be less of an issue in the long run, because in the Qt 6 time frame we will need to investigate better build system integration anyway, and so manually running tools like qsb will be less common.
Wait, where is this qsb thing coming from?
From the Qt Shader Tools module. This provides both an API, QShaderBaker, and a command-line host tool, qsb, to perform the compilation, translation, and packaging steps described above.
There is just one catch: this is a qt-labs module for the time being, so it does not ship with Qt 5.14.
Why is this? Well, mainly because of the third-party dependencies, like glslang and SPIRV-Cross. There are a number of things to investigate and figure out when it comes to being able to compile and run on all our target platforms, some things related to licenses, etc. If all this sounds familiar, it is because some of these issues were mentioned in part 1 of this blog series when we talked about API translation solutions. So for now generating a .qsb package involves checking out and building this module, and then running the qsb tool manually.
While we will need a solution that part of and ships with Qt, relying on offline shader processing is not such a bad thing. It is namely one of our targets for Qt 6 in any case. The vision is to have something that integrates with Qt's build system, so that the shader processing steps described above are done during application (or library) build time. This is left as a future exercise however, mainly because of the upcoming qmake -> cmake transition. Once things stabilize, we can start building a solution on top of the new system.
So how does Qt Quick do it in Qt 5.14?
Looking at qtdeclarative/src/quick/scenegraph/shaders_ng, the answer is obvious: by running qsb manually (notice the aptly named compile.bat) and including the resulting .qsb files in the Qt Quick library via the Qt resource system. This should become a bit more sophisticated later on, as outlined above, but does the job for now.
The .vert and .frag files contain Vulkan-compatible GLSL code and are not shipping in the Qt Quick build. Only the .qsb files are listed in scenegraph.qrc.
The process is nicely summed up in this slide borrowed from my NDC TechTown 2019 presentation:
Each material only has a single pair of vertex and fragment shaders, always written as Vulkan-compatible GLSL, following a few simple conventions (like only using a single uniform buffer, placed at binding 0).
Each of these files are then run through the shader baking machinery, resulting in a QShader package. In the example the result is 6 versions of the same shader, plus the reflection data (which qsb can print as JSON text; the .qsb files themselves are compressed binary files, however, and are not human readable). Problem 1 & 2 from above are thus solved.
Note the [Standard] tags in the shader list. Had this been a vertex shader, and the -b argument had been specified as well, the number of output shaders would have been 12, not 6. The 6 additional ones would have been marked as [Batchable], indicating those were the batching friendly, slightly modified variants targeted for the Qt Quick scenegraph's renderer. This solves problem 3, at the expense of slightly higher storage requirements. (but due to the reduced run time work this is likely worth it in the end)
This covers the core concepts behind the new shader pipeline. We have to look at ShaderEffect and QSGMaterial in a separate post. The basic idea (as of Qt 5.14) is to pass in .qsb file names instead of shader source strings, but materials in particular have to be aware a few more things (mainly due to working with uniform buffers instead of single uniforms, and due to not having the concept of a per-thread current context where anyone can change states arbitrarily). So more about all that another time.
Subscribe to our newsletter
Try Qt 6.0 Now!
Download the latest release here: www.qt.io/download.
Qt 6 was created to be the productivity platform for the future, with next-gen 2D & 3D UX and limitless scalability.
Check out all our open positions here and follow us on Instagram to see what it's like to be #QtPeople.
Näytä tämä julkaisu Instagramissa.
Want to build something for tomorrow, join #QtPeople today! We have loads of cool jobs you don’t want to miss! http://qt.io/careers #builtwithQt #software #developers #coding #framework #tool #tooling #C++ #QML #engineers #sales #tech #technology #UI #UX #CX #Qt #Qtdev #global #openpositions #careers #job
Henkilön Qt (@theqtcompany) jakama julkaisu