  Recent studies show evidence for emergent cognitive abilities in Large
Pre-trained Language Models (PLMs). The increasing cognitive alignment of these
models has made them candidates for cognitive science theories. Prior research
into the emergent cognitive abilities of PLMs has largely been path-independent
to model training, i.e., has focused on the final model weights and not the
intermediate steps. However, building plausible models of human cognition using
PLMs would benefit from considering the developmental alignment of their
performance during training to the trajectories of children's thinking. Guided
by psychometric tests of human intelligence, we choose four sets of tasks to
investigate the alignment of ten popular families of PLMs and evaluate their
available intermediate and final training steps. These tasks are Numerical
ability, Linguistic abilities, Conceptual understanding, and Fluid reasoning.
We find a striking regularity: regardless of model size, the developmental
trajectories of PLMs consistently exhibit a window of maximal alignment to
human cognitive development. Before that window, training appears to endow
"blank slate" models with the requisite structure to be poised to rapidly learn
from experience. After that window, training appears to serve the engineering
goal of reducing loss but not the scientific goal of increasing alignment with
human cognition.
