  Control policy learning for modular robot locomotion has previously been
limited to proprioceptive feedback and flat terrain. This paper develops
policies for modular systems with vision traversing more challenging
environments. These modular robots can be reconfigured to form many different
designs, where each design needs a controller to function. Though one could
create a policy for individual designs and environments, such an approach is
not scalable given the wide range of potential designs and environments. To
address this challenge, we create a visual-motor policy that can generalize to
both new designs and environments. The policy itself is modular, in that it is
divided into components, each of which corresponds to a type of module (e.g., a
leg, wheel, or body). The policy components can be recombined during training
to learn to control multiple designs. We develop a deep reinforcement learning
algorithm where visual observations are input to a modular policy interacting
with multiple environments at once. We apply this algorithm to train robots
with combinations of legs and wheels, then demonstrate the policy controlling
real robots climbing stairs and curbs.
