  Pre-trained transformer language models (TLMs) have recently refashioned
natural language processing (NLP): Most state-of-the-art NLP models now operate
on top of TLMs to benefit from contextualization and knowledge induction. To
explain their success, the scientific community conducted numerous analyses.
Besides other methods, syntactic agreement tests were utilized to analyse TLMs.
Most of the studies were conducted for the English language, however. In this
work, we analyse German TLMs. To this end, we design numerous agreement tasks,
some of which consider peculiarities of the German language. Our experimental
results show that state-of-the-art German TLMs generally perform well on
agreement tasks, but we also identify and discuss syntactic structures that
push them to their limits.
