  Reliable methods for automatic readability assessment have the potential to
impact a variety of fields, ranging from machine translation to self-informed
learning. Recently, large language models for the German language (such as
GBERT and GPT-2-Wechsel) have become available, allowing to develop Deep
Learning based approaches that promise to further improve automatic readability
assessment. In this contribution, we studied the ability of ensembles of
fine-tuned GBERT and GPT-2-Wechsel models to reliably predict the readability
of German sentences. We combined these models with linguistic features and
investigated the dependence of prediction performance on ensemble size and
composition. Mixed ensembles of GBERT and GPT-2-Wechsel performed better than
ensembles of the same size consisting of only GBERT or GPT-2-Wechsel models.
Our models were evaluated in the GermEval 2022 Shared Task on Text Complexity
Assessment on data of German sentences. On out-of-sample data, our best
ensemble achieved a root mean squared error of 0.435.
