  Commonsense reasoning benchmarks have been largely solved by fine-tuning
language models. The downside is that fine-tuning may cause models to overfit
to task-specific data and thereby forget their knowledge gained during
pre-training. Recent works only propose lightweight model updates as models may
already possess useful knowledge from past experience, but a challenge remains
in understanding what parts and to what extent models should be refined for a
given task. In this paper, we investigate what models learn from commonsense
reasoning datasets. We measure the impact of three different adaptation methods
on the generalization and accuracy of models. Our experiments with two models
show that fine-tuning performs best, by learning both the content and the
structure of the task, but suffers from overfitting and limited generalization
to novel answers. We observe that alternative adaptation methods like
prefix-tuning have comparable accuracy, but generalize better to unseen answers
and are more robust to adversarial splits.
