Over the past 5 years or so it has become abundantly apparent that automotive consumers expect that when they enter their car they are greeted by 2D controls that are not only responsive and intuitive, but also by 3D graphics that are functional and fluid.  From simple seat controls, to the complexities of ADAS scenes driven by real time vehicle sensor data, three-dimensional interfaces are paramount in assisting end users to understand their vehicle's status and its location on the roadway with respect to its surroundings.  Companies like Rivian and Toyota are leveling up their driving experience by leveraging 3D engines that were once delegated to strictly video game development.  The introduction of gamification to the automotive user experience has also caused a seismic shift in hardware, with many cars now requiring the GPU compute power to process vehicle sensor data and visualize it in 3D as well.  However, 3D rendering in automotive is not all strictly business; for example, Rivian's cel-shaded take on 3D is a playful one that hearkens back to their idyllic tagline: "Keep the world adventurous forever".  Ford is enhancing their user experience in the Mustang by using 3D to visualize vehicle modes, track mode, and even to create retro-inspired clusters.

Qt Quick 3D

Needless to say, 3D rendering in automotive has gone from a "nice-to-have" option to a prerequisite feature for any premium brand.  The team at Qt recognizes this industry requirement and has been enabling customers to enhance their Qt user interfaces with 3D for a number of years, most recently with QtQuick3D.  Qt Quick3D seamlessly brings the power of 3D rendering to your applications with minimal code, letting you transform 2D interfaces into immersive experiences without learning complex graphics APIs. Users can leverage Qt Quick's simple APIs to create stunning visualizations and engaging UIs while still maintaining the cross-platform compatibility Qt is known for.  Qt enables 3D development with their Qt Design Studio (pictured below) which exports projects to Qt Creator for building and deploying.

QtQuick3D is performant on embedded devices and has a visual fidelity similar to that of other mainstream game engines like Unreal and Unity.  Can you guess which engine is rendered in each image below (Qt, Unity or Unreal)?

3rd Party 3D Engine Integration

An oft-asked question here at Qt is: "how can I integrate a 3rd-party 3D engine into my Qt application?".  Qt helps to enable and even facilitate these type of architectures rather easy as of Qt 6, with the release of APIs like Qt ActivityView.  Soheil Armin, R&D Manager, Qt for Android Automotive recently described the API as: "a transformative new back-end for the ActivityView module, significantly advancing user interface development on Android Automotive. Moving beyond the limitations of traditional Java APIs offered by AOSP, this advanced back-end enables fluid integration of third-party Android apps directly into the Qt Quick applications. This brings a new level of flexibility to Qt for Android Automotive user interfaces."  Inspired by Soheil's blog post, I decided to test out this new API and see just how easy (or difficult) it might be to integrate an external 3D renderer into my Qt 2D interface. 

The genesis of this project began in Figma, where the 2D assets were created (Special thanks to Shawn Dorsey for the collaboration here):

As you can see above, there is special consideration given to leaving screen real estate for my 3D scene to the right.  From here, it was as simple as using the Qt Bridge for Figma plugin to autogenerate my QML code from my Figma design by opening the project export in Qt Design Studio:

At this point, I needed to switch out of my comfort zone and over to the Unity IDE.  With the help of a number of excellent YouTube creators, I was able to create the following scene:

This project was then built and installed to my AAOS device as a standalone APK aptly named: com.qt.MoonLander.

From here, I took the project from Qt Design Studio above and opened it in Qt Creator: 

To embed the Unity apk inside of my Qt application, I added the following code to my App.qml file:

ActivityView {
    id: activityView

    // package name of APK for 3D scene
    packageName: "com.qt.MoonLander"
    anchors.fill: parent

    // entry point for 2D Figma UI from Design Studio
    Screen01 {
        id: mainScreen
        anchors.centerIn: parent
    }
}

That's it.  The ActivityView is as simple and easy to use as any QML object and can be arranged on a screen in much the same way.  As you can see, the packageName could be anything, even an Unreal Engine scene.  Making the Screen01 qml object a child of the Unity activity view assures it is on the foreground.  From here, I signed, built and deployed the Android APK from Qt Creator to my Android Automotive OS 14 Emulator image.  Here it is in all it's beautiful simplicity:

You may notice when you watch this video that the top and bottom bar load early while the overlanding controls fade in nicely only after the vehicle scene is loaded.  This was an intentional design decision that I selfishly included to show one more powerful API from Qt for Android Automotive: AndroidBroadcastReceiver QML Type.  Intents are a longstanding Android interprocess communication protocol that enable you to easily communicate information between applications.  To enable this synchronization across domains was also extremely simple with the help of the aforementioned QML API:

// Add a new AndroidBroadcastReceiver for Unity splash complete
AndroidBroadcastReceiver {
    id: unitySplashReceiver
    AndroidIntentFilter {
        actions: [
            "com.qt.UNITY_SPLASH_COMPLETE" // The action we defined in Unity
        ]
    }

    onReceived: function(intent) {
        // Check for and log the timestamp extra if it exists
        if (intent.extras.hasOwnProperty("timestamp")) {
            console.log("Unity splash completed at:", intent.extras.timestamp);
        }

        // Inform the 2D UI that the Unity splash screen is complete
        // and now the controls can be shown
        mainScreen.splashScreenComplete = true;
    }
}

Note that Qt also provides ready-made, cross-platform integrations with a number of automotive-grade IPC protocols including: CAN, MQTT, Qt Remote Objects and more.

Striking a Balance Between Visuals and Safety

The integration of impressive 3D visuals in automotive displays must be carefully balanced with safety considerations. The ISO 26262 road vehicles safety standard requires freedom from interference between components of different ASIL ratings; while rich graphics can enhance driver understanding and create premium experiences, they must never compromise the reliable delivery of critical information. Safety-critical elements like speed indicators and warning notifications require guaranteed rendering even if decorative visual elements fail. A well-designed system implements proper isolation between these domains, ensuring that graphical ambitions don't interfere with the vehicle's primary safety functions. This delicate balance enables manufacturers to deliver both the wow-factor consumers expect and the dependability that regulatory requirements demand – ultimately creating interfaces that are both visually stunning and functionally uncompromising.  Qt Safe Renderer serves as the crucial middleware that bridges the gap between ASIL-rated safety requirements and rich user experiences. By implementing a monitor-renderer architecture with features like CRC checks and UI watchdogs, Qt ensures that critical vehicle information is displayed reliably even if the infotainment system experiences issues.  Here is a real-world example of one such architecture:

As you can see, Qt Safe Renderer can be used to render content in a domain that is "unsafe", but can then monitor that content from a domain that is "safe", such as an ASIL-rated microcontroller or safe core.  This content can then be combined with the other high-end visuals as shown above to orchestrate an architecture that is both visually stunning and safe.

Questions or thoughts?  Let's talk! Please reach out to me: taylor.fouts@qt.io

The full source code for this project can be found here: https://github.com/tbfouts/QtAndroidActivityView3D