  Compositionality has long been considered a key explanatory property
underlying human intelligence: arbitrary concepts can be composed into novel
complex combinations, permitting the acquisition of an open ended, potentially
infinite expressive capacity from finite learning experiences. Influential
arguments have held that neural networks fail to explain this aspect of
behavior, leading many to dismiss them as viable models of human cognition.
Over the last decade, however, modern deep neural networks (DNNs), which share
the same fundamental design principles as their predecessors, have come to
dominate artificial intelligence, exhibiting the most advanced cognitive
behaviors ever demonstrated in machines. In particular, large language models
(LLMs), DNNs trained to predict the next word on a large corpus of text, have
proven capable of sophisticated behaviors such as writing syntactically complex
sentences without grammatical errors, producing cogent chains of reasoning, and
even writing original computer programs -- all behaviors thought to require
compositional processing. In this chapter, we survey recent empirical work from
machine learning for a broad audience in philosophy, cognitive science, and
neuroscience, situating recent breakthroughs within the broader context of
philosophical arguments about compositionality. In particular, our review
emphasizes two approaches to endowing neural networks with compositional
generalization capabilities: (1) architectural inductive biases, and (2)
metalearning, or learning to learn. We also present findings suggesting that
LLM pretraining can be understood as a kind of metalearning, and can thereby
equip DNNs with compositional generalization abilities in a similar way. We
conclude by discussing the implications that these findings may have for the
study of compositionality in human cognition and by suggesting avenues for
future research.
