  Evolution has produced a multi-scale mosaic of interacting adaptive units.
Innovations arise when perturbations push parts of the system away from stable
equilibria into new regimes where previously well-adapted solutions no longer
work. Here we explore the hypothesis that multi-agent systems sometimes display
intrinsic dynamics arising from competition and cooperation that provide a
naturally emergent curriculum, which we term an autocurriculum. The solution of
one social task often begets new social tasks, continually generating novel
challenges, and thereby promoting innovation. Under certain conditions these
challenges may become increasingly complex over time, demanding that agents
accumulate ever more innovations.
