Shaping the Future of Digital Experience - UI Framework Performance

After discussing the UI framework's foundational elements, namely the set of ready-made solutions enabling rapid cross-platform development and efficient time-to-market of UI apps, we look here at what is needed to achieve the kind of smooth, seamless performance that defines the most engaging applications on the market. But let's start with a clarification of what is a "UI app" and how it differs from other types of visual software, like video games or design authoring tools.

The so-called “UI application”

While it is often taken for granted what a “UI application” is, there may be misconceptions about what shall be called a UI application and what cannot. Such misunderstanding may negatively impact our judgment in the choice of the right development framework for the creation of UI apps.

Consider graphics. It is a defining aspect of UI applications, but it is one aspect among many others. A UI application is not just graphics rendered on a screen—rather, it is a visual interface to live data, services, and functions that run in real time in the backend. Nor is it usually a standalone software that, while rendering a single scene on the entire screen, is allowed to consume all the available system resources—but a bundle of UI elements, like embedded apps and services, widgets, charts, interaction areas, and views, that coexist on one or multiple screens to provide users with the information they need for several different tasks in parallel. Several processes compete with one another in UI applications for the limited system resources—such as memory, processing power, storage, and often also power on low-energy devices—and we'll see some smart solutions the UI framework provides to handle concurrent tasks efficiently. But first, some more details on UI app structure and logic.

Layout

The UI layout is hierarchically structured, built from multiple layers and multiple apps occupying different portions of the screen. Touch and other forms of input should propagate through the UI layers until they reach the application or function they are intended for. Different types of interaction result in different outcomes: touching an icon or a button opens an app or service, swiping changes the view, zooming, scrolling are all triggered by specific actions on precise portions of the screen.

Additionally, UI applications are usually highly customizable and offer users the ability to change the overall look-and-feel, move widgets around, and play with the theme and layout of the UI, while preserving the overall intended functionality of the UI intact.

States

The UI behavior is organized in states identifying the different views with which the user can interact. The navigation across states defines the logic of the UI, namely what states are reachable from a given one (“from-to” transitions) and by what action. As the number of states can easily grow very large even in simple applications, it is essential to have flexible, reliable, and efficient tools to handle states and control the application flow, like those provided by a dedicated state machine framework.

Transitions

Visually, a sudden change from one view to another would be unpleasant, if not disturbing. To draw the user’s attention in a gentle, non-distracting way, easing functions are used to model how state transitions happen over time and display smooth, gradual transitions.

The list of the characteristic features of UI applications can grow longer. But it is clear that the internal structure, visual layout, and logic flow of UI applications are unique and radically different from those of other types of visual software like, for instance, design authoring tools or video games. As such, their proper and efficient management requires the dedicated tooling offered by the UI framework. 

Runtime

While on the development side, the UI framework provides libraries and solutions for app creation on PC, on the deployment side, it provides the UI engine running the applications on the end-user devices, also known as runtime. A smartwatch, a digital automotive cockpit, or the display of smart-home appliances all include an instance of the framework's UI engine running the app on their specific hardware and software architecture.

Considering the variety of such devices and their often very limited computational power and energy supply, it looks almost magic that, no matter the underlying system, a single runtime engine may control every function’s and service’s lifecycle, their interaction with one another, and the resource allocation for each task. But the UI engine is specifically designed to handle competing processes with particular attention to performance.

Performance

Digital devices are usually resource-constrained devices with very limited processing power and storage capabilities. The devices powering our appliances, cars, and wearables have, for instance, much more limited capabilities than the powerful desktop PCs used for home entertainment or games. Still, the final performance of the application and the quality of its graphics are not dictated solely by the hardware. The UI framework offers a series of ready-made solutions to optimize resource consumption in constrained environments, enabling a multitude of parallel services and advanced graphics with a low footprint.

Event-drive architecture

With UI software, it is rather exceptional that a single application is allowed to use all the resources available or to have unlimited startup time. The embedded devices used for customer electronics or for industrial, medical, and automotive applications need to snap-boot and have only scarce computational resources to run several functions and services in parallel. To contain resource usage, the UI framework provides a series of optimization mechanisms ensuring smooth performance and seamless responsiveness.

The underlying event-driven architecture uses events to trigger functionality and to communicate across decoupled services. Applications remain idle when not in use, minimizing the amount of memory, CPU, and GPU needed. This frees computational resources that can be dynamically allocated to other applications—and is also beneficial in terms of saving energy when the supply is limited. Dynamic memory loading and unloading goes hand-in-hand with UI virtualization that allows, roughly speaking, to load UI elements into memory only when needed—on-demand behavior.

In addition to UI virtualization, the UI framework supports:

• Background loading of UI and resources

• Threading to handle the concurrent execution of tasks across multiple CPU cores

• Memory management to automatically handle the allocation and deallocation of memory for objects

• Caching for images, fonts, and other resources, helping to speed up the loading and display of these resources

• Optimized data structures that are more efficient than the standard C++ data structures

• Compilation options include link-time optimization to find opportunities for optimization of the overall program.

The UI framework helps developers improve the efficiency and responsiveness of the UI by means of such mechanisms that streamline the UI's internal processes and reduce the memory footprint. But there is more to it.

Profiling

While a prototype can show the application’s intended behavior and look on a desktop PC—a sort of unconstrained environment that abstracts over the resource needs—rarely can such an application be ported to the actual target at this stage.

Profiling involves collecting data about how the application uses resources such as CPU, memory, and network bandwidth and analyzing this data to identify bottlenecks, lags, and memory leaks. Too complex 3D models, high-density shaders, extensive use of shader effects are frequent examples of such bottlenecks that may prevent an application from performing steadily at the optimal 60 frames per second. The profiler helps detect such rendering bottlenecks but also suboptimal code, background tasks that consume too many resources, memory leaks, unoptimized graphics, too frequent UI updates, and other similar issues that, without the right tools, would require a long time and unnecessary effort to be detected.

When the actual target hardware is not accessible, the UI framework's emulator allows developers to examine the app behavior on the target’s form factor and processing capabilities from their PC as if it was running on the actual device.

When the hardware is not yet agreed upon or needs to be re-considered (as in the case of chip shortage), the cross-platform UI framework ensures that software development and part of the testing can proceed unaffected with the least impact on time-to-market.

Quality assurance

If in addition to a fluid performance, the application needs to meet the safety and reliability industry standard and comply with the requirements set by notified bodies, the presence of integrated quality assurance tools helps automate the testing process, identify untested code, and detect code smells or deviance from coding directives. While in general beneficial to raise the quality and trust of the software product, such QA component becomes essential in case of safety-critical applications and certification thereof as, for instance, in the medical sector.

Our next post in the UI framework series will be about graphics and all the visual ingredients that bring UI applications to life. Stay tuned to learn how 2D and 3D graphics intertwine to create functional, compelling UI applications.