  Human learning is sensitive to rule-like structure and the curriculum of
examples used for training. In tasks governed by succinct rules, learning is
more robust when related examples are blocked across trials, but in the absence
of such rules, interleaving is more effective. To date, no neural model has
simultaneously captured these seemingly contradictory effects. Here we show
that this same tradeoff spontaneously emerges with ``in-context learning''
(ICL) both in neural networks trained with metalearning and in large language
models (LLMs). ICL is the ability to learn new tasks ``in context'' -- without
weight changes -- via an inner-loop algorithm implemented in activation
dynamics. Experiments with pretrained LLMs and metalearning transformers show
that ICL exhibits the blocking advantage demonstrated in humans on a task
involving rule-like structure, and conversely, that concurrent in-weight
learning reproduces the interleaving advantage observed in humans on tasks
lacking such structure.
