  Inspired by the necessity of morphological adaptation in animals, a growing
body of work has attempted to expand robot training to encompass physical
aspects of a robot's design. However, reinforcement learning methods capable of
optimizing the 3D morphology of a robot have been restricted to reorienting or
resizing the limbs of a predetermined and static topological genus. Here we
show policy gradients for designing freeform robots with arbitrary external and
internal structure. This is achieved through actions that deposit or remove
bundles of atomic building blocks to form higher-level nonparametric
macrostructures such as appendages, organs and cavities. Although results are
provided for open loop control only, we discuss how this method could be
adapted for closed loop control and sim2real transfer to physical machines in
future.
