  Recent work on evaluating the diversity of text generated by LLMs has focused
on word-level features. Here we offer an analysis of syntactic features to
characterize general repetition in models, beyond frequent n-grams.
Specifically, we define syntactic templates and show that models tend to
produce templated text in downstream tasks at a higher rate than what is found
in human-reference texts. We find that most (76%) templates in model-generated
text can be found in pre-training data (compared to only 35% of human-authored
text), and are not overwritten during fine-tuning processes such as RLHF. This
connection to the pre-training data allows us to analyze syntactic templates in
models where we do not have the pre-training data. We also find that templates
as features are able to differentiate between models, tasks, and domains, and
are useful for qualitatively evaluating common model constructions. Finally, we
demonstrate the use of templates as a useful tool for analyzing style
memorization of training data in LLMs.
