  Learning to locomote to arbitrary goals on hardware remains a challenging
problem for reinforcement learning. In this paper, we present a hierarchical
learning framework that improves sample-efficiency and generalizability of
locomotion skills on real-world robots. Our approach divides the problem of
goal-oriented locomotion into two sub-problems: learning diverse primitives
skills, and using model-based planning to sequence these skills. We parametrize
our primitives as cyclic movements, improving sample-efficiency of learning on
a 18 degrees of freedom robot. Then, we learn coarse dynamics models over
primitive cycles and use them in a model predictive control framework. This
allows us to learn to walk to arbitrary goals up to 12m away, after about two
hours of training from scratch on hardware. Our results on a Daisy hexapod
hardware and simulation demonstrate the efficacy of our approach at reaching
distant targets, in different environments and with sensory noise.
