The Qt 6.3.0 release is just around the corner, and I wanted to share some updates we have made for the Qt Wayland integration and its companion module Qt Wayland Compositor. I will also use this as an opportunity to introduce some basic concepts to those who are not already familiar with Wayland, and give some background/justification for how things currently work.
Basic introduction
For those who don't know, Wayland is a display server protocol that can be used as an alternative to X11 on Linux systems. The Qt Wayland Client module provides a way for a Qt application to run on such systems, and Qt Wayland Compositor provides a powerful API to write custom display servers from scratch.
The two modules together are what you need when you are creating a system based on Qt.
For context: Many Qt users are creating embedded devices where Qt is at the core of all user interaction. When such a system is sufficiently complex, it will often be to advisable to split it into multiple, separate processes. Isolating parts of the system in this way can make it both more secure and easier to maintain. But in order to make the conversion into a multi-process system, you need a way to share resources between the clients. Qt Wayland manages this for Qt-based systems.
Qt Wayland Compositor can also be used to create servers for desktop systems, of course. But the most common use case at the moment is for embedded systems, and some of the features described in this blog are things that may primarily be convenient in an embedded system where all applications are pre-defined and trusted.
For a more detailed introduction, check out the documentation.
Custom shell protocols
Shells are Wayland protocols which provide ways the client and server can communicate about surfaces (aka "windows" in a typical desktop context). This can be used for the "window manager" part of the server: Resizing surfaces, minimizing/maximizing them, defining their role in the system (is it a popup menu or an application window?) , et cetera.
Qt 6.2 (and earlier versions) supports three such protocols by default: WlShell, XdgShell and IviApplication. XdgShell is a "desktop-grade" shell, and provides the features you would expect on such a system. IviApplication is intended for IVI systems and is much simpler. (WlShell is a legacy protocol and exists for compatibility with older applications.)
When you are building a platform based on Qt, Wayland comes with some additional powerful tools: When both clients and server are running on the same framework, you can extend Wayland with your own protocols. This is a way to further customize how an application is integrated in the system. Qt Wayland and Qt Wayland Compositor are designed with this in mind, so adding extensions is both easy and convenient.
Before Qt 6.3, however, there was no public API for customizing the shell protocol of the application. Some users would still do it by hooking into the private API, but at the penalty of maintaining patches on top of Qt Wayland that had to be rebased when new versions were released.
In Qt 6.3, this gap has been filled, and writing custom shell extensions is now a supported use case. (It's worth noting that this is a rather advanced use case, and for most users the existing shells will probably be sufficient.)
In order to write a custom shell, you need to create an extension protocol, an implementation for the compositor, as well as a plugin that can be loaded by Qt applications. Together these pieces define how Qt application windows behave when running against your compositor, and you can run any unmodified Qt 6.3-based application with your shell extension as long as the compositor also supports it. For instance, in this screenshot, two instances of the particles3d example from Qt Quick 3D are running against the compositor. The code for the example has not been touched at all, and is the same that would be run on any other shell extension or platform.
For a full walk-through on how to do this, check out the Custom Shell Example.
Qt Shell Extension
The standard shell extensions such as XdgShell have built-in limitations that are designed to enforce a consistent and non-intrusive behavior on part of the client applications. For instance, a client is not allowed to forcibly activate itself, since this will risk interrupting what the user is currently doing, and be both confusing and frustrating.
The limitations exist for good reason, but they also limit the Qt APIs that can be supported. For instance, the QWindow::requestActivate() function is explicitly prohibited by this.
When building a custom platform consisting of trusted applications, the built-in limitations of XdgShell may be more in the way than they are useful. In some cases, it could be convenient to be able to use the existing APIs in Qt (such as QWindow::requestActivate()) rather than have to invent system-specific extensions for things like phone calls or warnings taking over the system.
For this purpose, we have implemented a special QtShell which supports all the window management features in Qt's APIs. It is not intended as a replacement for XdgShell, but only as an alternative that can be used under the special circumstances where the limitations of XdgShell and interoperability with third-party applications are not required.
The Qt Shell also comes with a customizable "Chrome" (the packaging around the window surface). Here you can add your own decorations and behavior to client windows (the Qt Shell only supports server-side decorations). The example has a very simple style to illustrate how window decorations can be customized, but they can easily be expanded in the code.
See the QtShell Compositor Example for details on how to use this shell in your own compositor.
Conclusion
Qt 6.3 introduces new ways of customizing the behavior and appearance of windows in your Qt Wayland-based systems. The standardized shells are still preferable - as long as they fulfill your requirements. But for some advanced use cases, it might be necessary to add features to the windowing system itself. Qt 6.3 adds two new ways of achieving this with Wayland.
