C++26 Reflection 💚 QRangeModel

In the Qt Company's R&D organization we have made it a tradition to start the year with a Hackathon where anyone can work on anything they find interesting. It's a great opportunity to work on or with something else than usual, to try out new technologies, or to generally scratch whatever might have been itching. We started with a pitching session the week before so that people that are looking for inspiration or projects to join know what's on the buffet. And then on Wednesday morning we kicked off the hacking, giving everyone two full days to work on their project before the presentations on Friday noon.

Cooking with QRangeModel

With the calendar having finally caught up with the upcoming C++ standard 26, I decided to play around with one of the most hyped feature of that new standard: reflections and annotations. Replacing the need for moc in 2 days seemed a bit ambitious, so instead I wanted to find out if we can teach our new QRangeModel to represent a range of plain C++ classes as a model that Qt Quick's item views can work with. That means, no metaobject from the Q_GADGET or Q_OBJECT macro, but also no tuple-protocol boiler plate. A list of

struct Entry
{
    QString name;
    double value;
};

should be represented as a two-column table model; the column names (as per QRangeModel's QAbstractItemModel::headerData implementation) should be "name" and "value". If explicitly tagged as a multi-role item, or if used in a table of such entries (such as QList<QList<Entry>>), then each item should have data for "name" and "value" roles.

model = std::make_unique(QList<QList<Entry>>{
    {
        {"zero", 0}, {"zero.one", 0.1}, {"zero.two", 0.2}
    },
    {
        {"one", 1}, {"one.one", 0.1}, {"one.two", 1.2}
    },
    {
        {"two", 2}, {"two.one", 2.1}, {"two.two", 2.2}
    },
});
</qlist

The QML code with a UI using a ListView that operates on that model would then be:

Rectangle {
    id: root
    visible: true
    implicitWidth: 1200
    implicitHeight: 500

    property AbstractItemModel model

    ListView {
        id: list
        model: root.model
        anchors.fill: parent
        delegateModelAccess: DelegateModel.ReadWrite
        delegate: Text {
            id: delegate
            required property var name
            required property var value

            width: ListView.view.width
            text: delegate.name + ": " + delegate.value
            MouseArea {
                anchors.fill: parent
                onClicked: ++delegate.value
            }
        }
    }
}

We should be able to access the data for the "name" and "value" role by named properties, and we want to be able to modify that data directly (i.e. increase the "value" by 1 when clicking on an entry). And all that without a single line of boiler plate or meta-object-compiler generated code!

As a stretch, I also wanted to see if we can go beyond simple aggregates, and support a type that's perhaps a bit more common in Qt code:

class Class
{
public:
    explicit Class() = default;

    explicit Class(const QString &name, double value)
        : m_name(name), m_value(value)
    {}

    QString name() const { return m_name; }
    void setName(const QString &name)
    {
        m_name = name;
    }

    double value() const { return m_value; }
    void setValue(double value)
    {
        m_value = value;
    }

private:
    QString m_name;
    double m_value = -1;
};

What kind of C++26 annotations could we use and reflect on to turn the above into an item type for a model?

Sharpening the knives

The project started by looking for the compiler sources of the ongoing implementations of C++26 in general, and reflection in particular. For gcc I found https://forge.sourceware.org/marek/gcc/src/branch/reflection to be the most up-to-date. For clang I found the p2996 branch of https://github.com/bloomberg/clang-p2996.git. But with only 2 days, and some build-system struggles when trying to getting clang to use the standard library headers from that branch, I decided to focus on using gcc on Linux.

By coincidence, the gcc feature branch merged into trunk on Thursday, so by now you get reflection support by building from trunk:

$ cd ~
$ git clone git://gcc.gnu.org/git/gcc.git gcc-trunk
$ mkdir gcc-trunk-build
$ cd gcc-trunk-build
$ ../gcc-trunk/configure --prefix=/opt/gcc-trunk --enable-languages=c,c++ --disable-multilib
$ make -j $(nproc)
$ sudo make install

To avoid any kind of trouble when mixing binaries built with different gcc versions, C++ standards, and standard library versions, I first built a fresh subset of Qt from the dev branch:

$ cd ~
$ git clone git://code.qt.io/qt/qt5.git qt-dev
$ cd qt-dev
$ ./init-repository -submodules qtdeclarative
$ cd -
$ mkdir qt-dev-build
$ cd qt-dev-build
$ ../qt-dev/configure -developer-build -no-warnings-are-errors -make tests -submodules qtdeclarative \
  -- -DCMAKE_C_COMPILER=/opt/gcc-trunk/bin/gcc -DCMAKE_CXX_COMPILER=/opt/gcc-trunk/bin/g++ -DQT_BUILD_TESTS_BY_DEFAULT=OFF
$ ninja -k 0

The init-repository invocation fetches all submodules that are required for building qtdeclarative. The configure line makes sure that I don't have to install Qt after building it (i.e. the prefix is $PWD), that newly introduced warnings don't stop the build, that all tests are configured, but not compiled. I only want to build qtdeclarative and everything required for that, but not leaf modules.

If the top of the configure output starts with

-- The CXX compiler identification is GNU 16.0.1
-- The C compiler identification is GNU 16.0.1

then we have done things correctly. The build will produce a number of new warnings from -Wsfinae-incomplete, which I didn't investigate.

Since I wanted to make sure that I'm starting the QRangeModel work from a working baseline, the test to run was obviously tst_qrangemodel:

$ ninja tst_qrangemodel_check

The test runs successfully. Time to get hacking.

Mise en place

The public QRangeModel class looks like a plain QAbstractItemModel implementation; only the constructor is a template. The bulk of the implementation however is done with the help of internal templates. It involves a collection of template specializations that allow the compiler to decide whether the rows and items of the model are ranges, arrays, types with meta object, or types for which the tuple protocol is implemented. This allows us in principle to use partial template specialization, and to provide a C++26 implementation for types that are plain aggregates.

Some of the logic however is currently written using if constexpr expressions. Those can't be specialized, but not even for a hackathon did I want to add any C++26 code to QRangeModel's implementation. At the very least, it would have slowed the work down if I'd have to build parts of Qt for every change, rather than just the test case. So after a few experiments that revealed the limitations of the current design, the next step was to extract some of the hardwired logic into specializable templates. The result was a small number of patches that refactor and clean up the respective implementation details.

After that work, I had customization points for accessing an element of a row, for getting the names of the column of a row, and for getting the list of role names for an item type. As a note: a bit more plumbing than what I'm showing here is involved to put all of this together; none of those QRangeModelDetails traits are meant to be public API or customization point. For the complete solution, see the chain of commits ending at https://codereview.qt-project.org/c/qt/qtbase/+/704327/11.

Specializing for Aggregates

The standard library provides us with a type trait std::is_aggregate. The definition of aggregates includes array types (including std::array, which std::is_array doesn't match), and the author of an aggregate type might also implement the tuple protocol. For both arrays and tuples we already specialize QRangeModel's access machinery, so to avoid conflicts, I narrowed this down to a "pure aggregates" concept:

template <typename T>
concept PureAggregate = std::conjunction_v<
    std::is_aggregate<T>,
    std::negation<std::is_array<T>>,
    std::negation<QRangeModelDetails::array_like<T>>,
    std::negation<QRangeModelDetails::tuple_like<T>>
>;

A specialization of the row_traits template would then look like this:

template <PureAggregate T>
struct row_traits<T>
{
    // ~~~
};

The row_traits type is responsible for telling QRangeModel about how many elements that row type has, and whether that number of elements is compile-time defined, a runtime-defined constant, or completely dynamic. And with the additions from the preparation work, it also has to provide the implementation of accessing an element if the size is compile-time defined.

My first attempt was to use only the new C++26 language feature of structured binding packs, P1061.

static consteval auto element_count()
{
    auto [...e] = T{};
    return sizeof...(e);
}

This doesn't work: our Entry class is not constexpr - QString allocates and frees memory, so constructing and destructing Entry in a consteval context is not possible. It would work with an Entry type that has only plain old data as members, but that's rather limited.

So, time to go all in with C++26 reflection, P2996, including the new meta library:

#include <meta>

// ...

static consteval auto element_count()
{
    return nonstatic_data_members_of(^^T, std::meta::access_context::current()).size();
}

Here we see one of the three new C++ syntax elements of C++26 in action: the reflection operator, ^^. It returns a reflection on it's operand, T (i.e. Entry) in the form of a std::meta::info object. This is an opaque, compiler-defined entity. The std::meta library then provides a number of consteval functions that take such an info object to return compile-time information about the reflected type. In this case, we are interested in those non-static data members of Entry that are accessible from the current context, and we only want the number of those elements. Note that we don't have to fully qualify nonstatic_data_members_of as std::meta::nonstatic_data_members_of - thanks to argument-dependent lookup, the compiler will look for that function the same namespace as its std::meta::info argument, which is always going to be std::meta.

Now we have an implementation of the element count that also works also for our Entry type.

The next step is to access the right data member of Entry when QRangeModel wants to read from or write to them. The row_traits function that does that is called for_element_at, and it gets called by QRangeModel with a lambda that does the framework bits - return the value as a QVariant when reading, or emit the dataChanged() signal when writing. We need to call that function with a reference to the element.

Maybe that's where a structured binding pack works:

template <typename Row, typename F>
static auto for_element_at(Row &&row, std::size_t idx, F &&function)
{
    auto &[...e] = &row;
    function(std::forward_like<Row>(e...[idx]));
}

It does not: the indexing operator of a structured binding pack has to be a constant expression, and idx is a runtime parameter. We could write a switch statement by hand, or we could let the compiler generate the equivalent of such a switch, using a fold expression. Since we needed that a number of times in the QRangeModel implementation, we have a internal little helper that we can reuse:

template <typename Row, typename F>
static auto for_element_at(Row &&row, std::size_t idx, F &&function)
{
    auto &[...e] = row;
    QtPrivate::applyIndexSwitch<sizeof...(e)>(idx, [&](auto idxConstant) {
        function(std::forward_like<Row>(e...[idxConstant.value]));
    });
}

We can now read from and write to any data member of an aggregate type like Entry. For that we can use the structured binding pack, with std::forward_like from C++23 to make sure that we pass the element to the function with the same value category as we got the forwarding reference to row. So if row is a const reference, then we pass the element as a const reference as well.

The last responsibility of the row_traits specialization (after the refactoring mentioned above) is to provide the name of each column as a QVariant (holding, in practice, a QString). This time we use the std::meta library facilities again, now to get the identifier of the element at an index.

    static QVariant column_name(int section)
    {
        QVariant result;
        QtPrivate::applyIndexSwitch<element_count()>(section, [&](auto idxConstant) {
            constexpr auto member = nonstatic_data_members_of(^^T,
                                        std::meta::access_context::current()).at(idxConstant.value);
            result = QString::fromUtf8(u8identifier_of(member).data());
        });
        return result;
    }

We can now treat our Entry type as a row with two columns, and a QRangeModel model(QList<Entry> {...}) in a QTableView will give us that: two columns, with titles name and value, showing the respective value for each row. And we can even edit each value.

The Qt Quick UI however wants to access the data as roles, not as columns. Before that works we we need to add a bit more specialization. We need to specialize the item_traits and implement a roleNames function that returns the names of the elements in a QHash<int, QByteArray> that maps from Qt::ItemDataRole to the name of that role. That is more more or less the same code as our column_name implementation, so I'm not repeating it here.

Lastly, we need to actually implement the access logic, this time by Qt::ItemDataRole, not by index. Qt 6.11 introduces a template prototype QRangeModel::ItemAccess that type authors can specialize to implement custom read and write operations of their type. That's a public customization point, so we can just use that. The QRangeModel convention for Qt gadgets or objects is then that custom properties are mapped to Qt::UserRole + n.

template <QRangeModelDetails::PureAggregate T>
struct QRangeModel::ItemAccess<T>
{
    using row_traits = QRangeModelDetails::row_traits<T>;

    static QVariant readRole(const T &item, int role)
    {
        const int index = role - Qt::UserRole;
        if (index < 0 || index >= row_traits::element_count())
            return {};

        QVariant result;
        QtPrivate::applyIndexSwitch<row_traits::element_count()>(index, [&](auto idxConstant){
            constexpr auto member = nonstatic_data_members_of(^^T,
                                        std::meta::access_context::current()).at(idxConstant.value);
            result = item.[:member:];
        });
        return result;
    }

The read-access implementation uses our applyIndexSwitch helper again, and then uses the second operator that C++26 introduces to the C++ language: the splice operator [: :]. Given the reflection member of a data member of the type T (which item is an instance of), item.[::] splices an access of that member into the C++ code. So for index 0 this generates item.name and for index 1 this generates item.value.

Writing a QVariant to that member is a bit more tricky, as we need to extract the value with the correct type from the QVariant. Since we know which member we want to access, we have two options to deduce that type: decltype(item.[:member:]), or by splicing the result of the the type_of(member) function of the meta library into the code:

    static bool writeRole(T &item, const QVariant &data, int role)
    {
        const int index = role - Qt::UserRole;
        if (index < 0 || index >= row_traits::element_count())
            return {};
        bool result = false;
        QtPrivate::applyIndexSwitch<row_traits::element_count()>(index, [&](auto idxConstant){
            constexpr auto member = nonstatic_data_members_of(^^T,
                                        std::meta::access_context::current()).at(idxConstant.value);
            using MemberType = [:type_of(member):];
            result = data.canConvert<MemberType>();
            if (result)
                item.[:member:] = data.value<MemberType>();
        });
        return result;
    }
};

To avoid that QRangeModel splays the Entry items in a QList<Entry> into two columns, we can QRangeModel::RowOptions for our Entry type:

template <>
struct QRangeModel::RowOptions<Entry>
{
    static constexpr auto rowCategory = QRangeModel::RowCategory::MultiRoleItem;
};

If we now use that QRangeModel(QList<Entry>) as a model with our QML UI, then we'll see that the list is rendered with each item showing name: value. If we'd implement the ItemAccess::readRole specialization for Qt::DisplayRole, then we'd also see something in a widget QListView or QTableView.

Reflection with Annotations

When I got to this point it was still Wednesday, so plenty of time to see if we can make the second type of item work: a type that doesn't qualify as an aggregate, with encapsulation, member access functions, and user defined constructors.

First, we need to identify some unique aspect of such types that allow us to specialize the templates. To get things going quickly, and because it's anyway a bit verbose that we have to specialize the QRangeModel::RowOptions template to tag a type as a multi-role item, I decided to try to use C++26 annotations for reflection, P3394. This introduces us to the third new syntax element of C++26, [[=...]]. It looks very similar to attributes, but comes with more flexibility.

template <QRangeModel::RowCategory Category>
class [[=Category]] Class
{
    // ~~~
};

Any "structural type" can be used as an annotation. Any type that can be used as a non-type template parameter can also be used in an annotation, so the best C++ reference documentation that I found was the page about template parameter. So we can use the values from an enum such as QRangeModel::RowCategory as an annotation, which allows us to define a concept based on such an annotation being present:

namespace QRangeModelDetails
{
template <typename T>
static consteval std::optional<QRangeModel::RowCategory> rangemodel_category()
{
    auto categories = annotations_of_with_type(^^T, ^^QRangeModel::RowCategory);
    if (categories.size())
        return extract<QRangeModel::RowCategory>(categories.front());
    return std::nullopt;
}

template <typename T>
concept RangeModelElement = rangemodel_category<T>().has_value();

template <RangeModelElement T>
struct row_traits<T>
{
    // ~~~
}
}

Our row_traits specialization will be used for any type T that has at least one annotation of the type QRangeModel::Category.

Now we need an equivalent of our element_count implementation for this type. This time we want to count non-const member functions. But since a type will have such functions that are not properties, and since we just learned about annotations, we might perhaps want to be a bit more explicit. Let's introduce another enum that we can use:

namespace Qt { // yay hackathon!
enum class Property {
    Readable    = 0x0000,
    Writable    = 0x0000,
    Final       = 0x0002,
};
}

Sadly, I can't use Q_DECLARE_FLAGS and Q_DECLARE_OPERATORS_FOR_FLAGS to get a property Qt flag type, because QFlag has a private member i, so it's not a structural type. For now, let's not care about that, we can nevertheless annotate member functions:

template <QRangeModel::RowCategory Category>
class [[=Category]] Class
{
public:
    // ~~~

    [[=Qt::Property{}]] QString name() const { return m_name; }
    void setName(QString name)
    {
        m_name = name;
    }
};

Let's say that any member of a type is a property getter if it's a const function that is annotated by a Qt::Property value:

template <RangeModelElement T>
struct row_traits<T>
{
    static consteval bool is_property_getter(std::meta::info member)
    {
        return is_const(member)
            && is_function(member)
            && annotations_of_with_type(member, ^^Qt::Property).size();
    }

We can then count how many of those we have using std::ranges::count_if:

    static consteval std::size_t property_count()
    {
        return std::ranges::count_if(members_of(^^T, std::meta::access_context::current()),
                                     is_property_getter);
    }

And we can get the reflection of such a member by index:

    static consteval std::optional<std::meta::info> property_getter(std::size_t idx)
    {
        for (std::meta::info member : members_of(^^T, std::meta::access_context::current())) {
            if (is_property_getter(member) && !(idx--))
                return member;
        }
        return std::nullopt;
    }

The column name would then be the name of that getter:

    static QVariant column_name(std::size_t section)
    {
        QVariant result;
        QtPrivate::applyIndexSwitch<property_count()>(section, [&](auto idx) {
            constexpr auto member = property_getter(idx);
            if constexpr (member)
                result = QString::fromUtf8(u8identifier_of(*member).data());
        });
        return result;
    }

And we can call that getter function on our row object to use it as a column value:

    template <typename F>
    static auto for_element_at(const T &row, std::size_t column, F &&function)
    {
        QtPrivate::applyIndexSwitch<property_count()>(column, [&](auto idx){
            constexpr auto member = property_getter(idx);
            if constexpr (member)
                function(row.[:*member:]());
        });
    }

Note the difference here: we are not using a forwarding reference for the Row type, we only implement the for_element_at function for the read-access case. For the write access, we need to solve two problems: finding the setter if we know the getter, and hooking a call to that setter into the machinery that implements QRangeModel::setData. The latter required a small addition to the QRangeModel implementation, allowing me ultimately to pass a functor object back into the machinery, and that functor object can then call the property setter. I'll spare you the details, as this is definitely a hack; the relevant lines of code are:

    struct CallSetter
    {
        bool operator()(const QVariant &data)
        {
            bool result = false;
            QtPrivate::applyIndexSwitch<property_count()>(column, [this, &result, &data](auto idx){
                constexpr auto member = property_setter(idx);
                if constexpr (member) {
                    constexpr auto parameter = parameters_of(*member).at(0);
                    using value_type = std::remove_cvref_t<decltype([:parameter:])>;
                    if (data.canConvert<value_type>()) {
                        row->[:*member:](data.value<value_type>());
                        result = true;
                    }
                }
            });
            return result;
        }

        T *row;
        std::size_t column;
    };

    template <typename F>
    static auto for_element_at(T &row, std::size_t column, F &&function)
    {
        return std::forward<F>(function)(CallSetter{&row, column});
    }

Given that we have a property_setter that returns the member function of T for the property we have at index column, get the type of the first parameter, and check if we can get such a value out of the QVariant. If so, call the setter with that value, and return success; otherwise return failure.

Which leaves the last puzzle piece: if we know that we have a property name, how do we find the reflection for the member function setName (or set_name, if snake-case is your thing)? We look for a non-const member function with a matching name:

    static consteval std::optional<std::meta::info> property_setter(std::size_t idx)
    {
        std::optional<std::meta::info> getter = property_getter(idx);
        if (getter && has_identifier(*getter)) {
            auto property_name = identifier_of(*getter);
            const auto set_prefix = std::string("set");
            for (std::meta::info member : members_of(^^T, std::meta::access_context::current())) {
                if (has_identifier(member) && is_function(member) && !is_const(member)) {
                    if (identifier_of(member) == set_prefix + "_" + property_name) {
                        return member;
                    } else {
                        auto setter_name = set_prefix + property_name;
                        auto &first = setter_name[3];
                        if (first >= 'a' && first <= 'z') // poor-man's compile-time uppercase
                            first -= ' ';
                        if (identifier_of(member) == setter_name)
                            return member;
                    }
                }
            }
        }
        return std::nullopt;
    }

Now we have all the pieces in place. With a bit more plumbing to hook all that logic into the framework, we don't even need to specialize QRangeModel::RowOptions. And we have a value type that is a template, and nevertheless can have a property getter and setter.

Conclusion

By Friday demo-time things were working as hoped, and I had a lot of fun playing with those new language features, even though the documentation is still sparse, lacking examples, and some of the Compiler Explorer projects the papers link to no longer compile with the latest revisions of the APIs.

C++26 will become an official ISO standard in March 2026. The implementation in gcc has worked very well, kudos to Marek Polacek and Jakub Jelinek, and the entire gcc team participating in the development and reviews! I'm sure that the implementation for clang will not be far behind, and that Microsoft has something cooking as well.

Also thanks to the C++ standard committee who have been listening to our feedback to earlier versions of the reflection proposal. The QRangeModel use case was fun to play with reflections, and perhaps we are able to merge some of the code from this hackathon into upstream once the standard is published: it's all header-only, so if you don't have a reflection-capable compiler, then it won't see those specializations and you can't use plain aggregates or annotated types as done here. But if you can upgrade to the latest, then these capabilities will be at your disposal, at least as experimental technology preview.

The obvious question is then if and how we plan to use C++26 reflections to replace moc. I have not done a feature-by-feature comparison between the meta object data we need to generate, and what we can get out of std::meta; but it seems that we can make the C++ compiler do much of the work that moc does. The biggest challenge might be the signals: and slots: member function blocks; we might have to annotate every function separately.

Thanks for reading this all the way to the end. Time to get back to finishing Qt 6.11, and some of the new things we are about to ship - including a few more QRangeModel features and classes, of course! Feedback is very welcome - are you using QRangeModel already? And do you think that C++26 provides compelling reasons to upgrade your compile tool chain?