  We present a framework for learning to plan hierarchically in domains with
unknown dynamics. We enhance planning performance by exploiting problem
structure in several ways: (i) We simplify the search over plans by leveraging
knowledge of skill objectives, (ii) Shorter plans are generated by enforcing
aggressively hierarchical planning, (iii) We learn transition dynamics with
sparse local models for better generalisation. Our framework decomposes
transition dynamics into skill effects and success conditions, which allows
fast planning by reasoning on effects, while learning conditions from
interactions with the world. We propose a simple method for learning new
abstract skills, using successful trajectories stemming from completing the
goals of a curriculum. Learned skills are then refined to leverage other
abstract skills and enhance subsequent planning. We show that both conditions
and abstract skills can be learned simultaneously while planning, even in
stochastic domains. Our method is validated in experiments of increasing
complexity, with up to 2^100 states, showing superior planning to classic
non-hierarchical planners or reinforcement learning methods. Applicability to
real-world problems is demonstrated in a simulation-to-real transfer experiment
on a robotic manipulator.
