  Transformers can under some circumstances generalize to novel problem
instances whose constituent parts might have been encountered during training
but whose compositions have not. What mechanisms underlie this ability for
compositional generalization? By reformulating multi-head attention as a
hypernetwork, we reveal that a low-dimensional latent code specifies key-query
specific operations. We find empirically that this latent code is highly
structured, capturing information about the subtasks performed by the network.
Using the framework of attention as a hypernetwork we further propose a simple
modification of multi-head linear attention that strengthens the ability for
compositional generalization on a range of abstract reasoning tasks. In
particular, we introduce a symbolic version of the Raven Progressive Matrices
human intelligence test on which we demonstrate how scaling model size and data
enables compositional generalization and gives rise to a functionally
structured latent code in the transformer.
