  Does inverse scaling only occur as a function of model size, or can it also
occur over the course of training? We carry out an exploratory study
investigating whether the performance of language models on specific tasks can
decrease (while general performance remains high) during training on the
language modeling task. We find 8 tasks on which Pythia 12B (Biderman et al.,
2023) shows decreased performance over the course of training. Five of these
tasks (TruthfulQA-MC1, TruthfulQA-MC2, Hindsight Neglect, Memo Trap, and
Pattern Match Suppression) additionally show a consistent relationship whereby
larger language models show a greater decrease in performance the more they are
trained, despite showing standard (positive) scaling overall. This highlights
the importance of testing performance at all relevant benchmarks any time
models are trained on additional data, even if their overall performance
improves
