  A hallmark of human intelligence is the ability to construct self-contained
chunks of knowledge and adequately reuse them in novel combinations for solving
different yet structurally related problems. Learning such compositional
structures has been a significant challenge for artificial systems, due to the
combinatorial nature of the underlying search problem. To date, research into
compositional learning has largely proceeded separately from work on lifelong
or continual learning. We integrate these two lines of work to present a
general-purpose framework for lifelong learning of compositional structures
that can be used for solving a stream of related tasks. Our framework separates
the learning process into two broad stages: learning how to best combine
existing components in order to assimilate a novel problem, and learning how to
adapt the set of existing components to accommodate the new problem. This
separation explicitly handles the trade-off between the stability required to
remember how to solve earlier tasks and the flexibility required to solve new
tasks, as we show empirically in an extensive evaluation.
