Back to Blog home

Qt Virtual Keyboard Updated with Handwriting Recognition

Published on Wednesday March 02, 2016 by Rainer Keller in Embedded Dev Loop | Comments

We are proud to present the new release of Qt Virtual Keyboard with Hand Writing Recognition (HWR), performance improvements, Nuance T9 Write integration, and support for Traditional Chinese language!

Qt Virtual Keyboard is now updated with new features and versioning scheme. As part of the recent licensing change announcement, the formerly commercial-only feature, the Qt Virtual Keyboard is now available also with GPLv3 license for open-source users, in addition to commercial Qt licensees. We released a technology preview of Qt Virtual Keyboard 2.0 a while back, and have now been improving it based on the feedback received. We have also adapted a new version numbering scheme: the Qt Virtual Keyboard now follows Qt versions. With the nupcoming Qt 5.6, Virtual Keyboard is still packaged only to the commercial installers, open source users need to get it from the repository. From Qt 5.7 onwards, Qt Virtual Keyboard is also included in the open source installers.

Qt Virtual Keyboard is a fully comprehensive out-of-the-box input solution. The most important new features of the Qt Virtual Keyboard include:

  • A reference implementation of handwrite input method with Lipi toolkit alphabetic + numeric recognizer integration (English)
  • Performance optimizations for Lipi toolkit
  • Accelerated processing of HWR recognition results
  • Full screen HWR mode (on-top writing)
  • Nuance T9 Write HWR integration
  • Word reselect for Hunspell word prediction
  • Support for runtime language switching (from the application)
  • Traditional Chinese keyboard layout

Check out the following video to see the latest version of the Qt Virtual Keyboard in action:

https://www.youtube.com/watch?v=yIsQHinTw00

Full screen HWR mode (on-top writing)

In the technology preview, the HWR integration used only the regular keyboard layout as HWR input area. In the new fullscreen HWR mode, which can be used in addition, the whole screen is used as an input area instead. Fullscreen HWR mode can be activated from the keyboard by double tapping the full screen HWR button. When the fullscreen HWR mode is active the keyboard is hidden. Trace input is activated and inactivated from a floating button on screen. Fullscreen HWR mode is then deactivated by double tapping the floating button. The switching from writing to selection mode and back is done by single tapping the floating button.

Handwriting in fullscreen mode

Performance optimizations for Lipi toolkit

We are using Lipi toolkit as an open source alternative handwriting recognition engine. Based on the technology preview, we found out that it did not perform well enough on low-end hardware. One obvious reason for this is that the Lipi toolkit was not optimized to run on embedded devices. We conducted some analysis and the performance is now improved by code-level optimization. With that, we have improved the performance by a neat 10-40% in recognition and data model loading!

Accelerated processing of recognition results

The Qt Virtual Keyboard runs the HWR tasks in a separate background thread. This allows the UI thread to continue its operation while the HWR tasks are running. But the recognition results could be produced even faster by starting the recognition already while waiting for the timeout of the user input.

Nuance T9 Write HWR integration

Nuance T9 Write is a commercial HWR engine, which can be enabled at compile time, if the user has a suitable license from Nuance. It is much faster than the Lipi toolkit on embedded hardware. Nuance T9 Write engine is integrated to Qt Virtual Keyboard as an alternative recognition engine for HWR mode. The initial Nuance T9 Write integration supports Latin languages, and is implemented in such a way that it is easy to support also the non-Latin languages in future releases. The integration can also be used with the on-top writing mode. Current support contains x86 and ARM targets.

Word reselect for Hunspell

Added word reselect feature for Hunspell input method. The word reselect allows the user to activate predictions / spell corrections for an existing word by touching the word in the input field. Before it was not possible to reselect a word.

Support for runtime language switching from the application

Earlier versions only supported to switch the language by pressing the language button on the virtual keyboard but there was no way to change the language from the program side. Because Qt API does not provide a uniform mechanism to change the input language, the QInputMethod API provides the current input locale as read-only property. Now the virtual keyboard settings API is extended to include options for controlling input language.
  • New properties locale, availableLocales and activeLocales are added to settings
  • The locale(if defined) overrides the system default locale in startup
  • The locale property can also change the runtime language
  • The availableLocales property is read-only and provides the list of “installed” locales
  • The activeLocales is application defined subset of availableLocales specifying the locales which can be activated runtime. If the list is empty, then all the available locales are active

Traditional Chinese

Added support for Traditional Chinese / Cangjie input method. The input method implementation is ported from Apache 2.0 licensed 3rd party library.
The Qt Virtual Keyboard supports 3 different Chinese input methods:
  • Pinyin(Chinese simplified)
  • Cangjie
  • Zhuyin

The type of available input methods is configured at compile time.

[gallery columns="1" type="slideshow" ids="44033,44034,44035"]

Get Qt Virtual Keyboard
Qt Virtual Keyboard is included in the commercial Qt 5.6 packages, Qt 5.6.0 final being released later in March. It has now also been contributed to open source under GPLv3 license by The Qt Company, and will be part of the Qt 5.7 release packages. If you are an open source user, please check the code from the repository. If you already have a commercial license, you can choose to install the new Qt Virtual Keyboard with handwriting support in conjunction with Qt 5.6.

Subscribe to Our Blog

Stay up to date with the latest marketing, sales and service tips and news.