Multi-process embedded systems with Qt for Device Creation and Wayland

With the Qt 5.4 update for Qt for Device Creation it is now possible - on certain embedded systems - to run Qt applications on top of a Wayland compositor by relying only on the provided reference images without any additional modifications. While the stable and supported approach remains eglfs, the lightweight platform plugin that allows running fullscreen Qt applications on top of EGL and fbdev with the best possible performance, those who do not require any of the enhanced tooling but need to have multiple GUI applications running on the same screen can start experimenting with a Wayland-based system already today.

In this post we will take a look how this can be done on i.MX6 based systems, like for example the Sabre SD and BD-SL-i.MX6 boards.

The wayland platform plugin provided by the Qt Wayland module is now an official part of Qt 5.4.0. Whenever the necessary dependencies, like the wayland client libraries and the wayland-scanner utility, are available in the sysroot, the platform plugin will be built together with the rest of Qt. In the toolchains and the ready-to-be-flashed reference images for Sabre Lite and Sabre SD everything is in place already. They contain Wayland and Weston 1.4.0, based on Yocto's recipes (daisy release).

We will use Weston as our compositor. This provides a desktop-like experience out of the box. Those looking for a more customized experience should look no further than the Qt Compositor libaries of the Qt Wayland module which provide the building blocks for easily creating custom compositors with Qt and QML. These components are still under development so stay tuned for more news regarding them in the future.

Video playback and some other applications running on a Sabre SD board

Now let's see what it takes to run a compositor and our Qt applications on top of it on an actual device. It is important to note that there is no tooling support for such a setup at the moment. This means that deploying and debugging from Qt Creator will likely not function as expected. Some functionality, like touch input and the Qt Virtual Keyboard, will function in a limited manner. For example, the virtual keyboard will appear on a per-application, per-window basis instead of being global to the entire screen. Support for such features will be improved in future releases. For the time being performance and stability may also not be on par with the standard single-process offering. On the positive side, advanced features like accelerated video playback and Qt WebEngine are already functional.

  • Qt Enterprise Embedded's reference images will launch a Qt application upon boot. This is either /usr/bin/qtlauncher, containing various demos, or the user's custom application deployed previously via Qt Creator. The launch and lifetime of these applications is managed by the appcontroller utility. To kill the currently running application, log in to the device via adb (adb shell) or ssh (ssh root@device_ip) and run appcontroller --stop.

  • Now we can launch a compositor. For now this will be Weston. The environment variable XDG_RUNTIME_DIR may not be set so we need to take care of that first: export XDG_RUNTIME_DIR=/var/run followed by weston --tty=1 --use-gal2d=1 &. The --use-gal2d=1 option makes Weston perform compositing via Vivante's hardware compositing APIs and the GC320 composition core instead of drawing textured quads via OpenGL ES.

  • Once the "desktop" has appeared, we are ready to launch clients. The default Qt platform plugin is eglfs, this has to be changed either via the QT_QPA_PLATFORM environment variable or by passing -platform wayland to the applications. The former is better in our case because we can then continue to use appcontroller to launch our apps. Let's run export QT_QPA_PLATFORM=wayland followed by appcontroller --launch qtlauncher. The --launch option disables some of the tooling support and will make sure a subsequent application launch via appcontroller will not terminate the previous application, as is the case with the default, eglfs-based, single GUI process system.

  • At this point a decorated window should appear on the screen, with the familiar demo application running inside. If the window frame and the title bar are not necessary, performance can be improved greatly by disabling such decorations altogether: just do export QT_WAYLAND_DISABLE_WINDOWDECORATION=1 before launching the application. Note that the window position can still be changed by connecting a keyboard and mouse, and dragging with the Windows/Command key held down.

To summarize:

appcontroller --stop
export XDG_RUNTIME_DIR=/var/run
weston --tty=1 --use-gal2d=1 &
export QT_QPA_PLATFORM=wayland
appcontroller --launch qtlauncher

and that's it, we have successfully converted our device from a single GUI app per screen model to a desktop-like, multi-process environment. To see it all in action, check out the following video:

Blog Topics: