  Real-world robotic manipulation tasks remain an elusive challenge, since they
involve both fine-grained environment interaction, as well as the ability to
plan for long-horizon goals. Although deep reinforcement learning (RL) methods
have shown encouraging results when planning end-to-end in high-dimensional
environments, they remain fundamentally limited by poor sample efficiency due
to inefficient exploration, and by the complexity of credit assignment over
long horizons. In this work, we present Efficient Learning of High-Level Plans
from Play (ELF-P), a framework for robotic learning that bridges motion
planning and deep RL to achieve long-horizon complex manipulation tasks. We
leverage task-agnostic play data to learn a discrete behavioral prior over
object-centric primitives, modeling their feasibility given the current
context. We then design a high-level goal-conditioned policy which (1) uses
primitives as building blocks to scaffold complex long-horizon tasks and (2)
leverages the behavioral prior to accelerate learning. We demonstrate that
ELF-P has significantly better sample efficiency than relevant baselines over
multiple realistic manipulation tasks and learns policies that can be easily
transferred to physical hardware.
