  Contemporary sensorimotor learning approaches typically start with an
existing complex agent (e.g., a robotic arm), which they learn to control. In
contrast, this paper investigates a modular co-evolution strategy: a collection
of primitive agents learns to dynamically self-assemble into composite bodies
while also learning to coordinate their behavior to control these bodies. Each
primitive agent consists of a limb with a motor attached at one end. Limbs may
choose to link up to form collectives. When a limb initiates a link-up action,
and there is another limb nearby, the latter is magnetically connected to the
'parent' limb's motor. This forms a new single agent, which may further link
with other agents. In this way, complex morphologies can emerge, controlled by
a policy whose architecture is in explicit correspondence with the morphology.
We evaluate the performance of these dynamic and modular agents in simulated
environments. We demonstrate better generalization to test-time changes both in
the environment, as well as in the structure of the agent, compared to static
and monolithic baselines. Project video and code are available at
https://pathak22.github.io/modular-assemblies/
