  Progress in the realisation of reliable large-scale quantum computers has
motivated research into the design of quantum machine learning models. We
present Quixer: a novel quantum transformer model which utilises the Linear
Combination of Unitaries and Quantum Singular Value Transform primitives as
building blocks. Quixer operates by preparing a superposition of tokens and
applying a trainable non-linear transformation to this mix. We present the
first results for a quantum transformer model applied to a practical language
modelling task, obtaining results competitive with an equivalent classical
baseline. In addition, we include resource estimates for evaluating the model
on quantum hardware, and provide an open-source implementation for classical
simulation. We conclude by highlighting the generality of Quixer, showing that
its parameterised components can be substituted with fixed structures to yield
new classes of quantum transformers.
