  Modern Reinforcement Learning (RL) algorithms promise to solve difficult
motor control problems directly from raw sensory inputs. Their attraction is
due in part to the fact that they can represent a general class of methods that
allow to learn a solution with a reasonably set reward and minimal prior
knowledge, even in situations where it is difficult or expensive for a human
expert. For RL to truly make good on this promise, however, we need algorithms
and learning setups that can work across a broad range of problems with minimal
problem specific adjustments or engineering. In this paper, we study this idea
of generality in the locomotion domain. We develop a learning framework that
can learn sophisticated locomotion behavior for a wide spectrum of legged
robots, such as bipeds, tripeds, quadrupeds and hexapods, including wheeled
variants. Our learning framework relies on a data-efficient, off-policy
multi-task RL algorithm and a small set of reward functions that are
semantically identical across robots. To underline the general applicability of
the method, we keep the hyper-parameter settings and reward definitions
constant across experiments and rely exclusively on on-board sensing. For nine
different types of robots, including a real-world quadruped robot, we
demonstrate that the same algorithm can rapidly learn diverse and reusable
locomotion skills without any platform specific adjustments or additional
instrumentation of the learning setup.
