  We describe a framework for building abstraction hierarchies whereby an agent
alternates skill- and representation-acquisition phases to construct a sequence
of increasingly abstract Markov decision processes. Our formulation builds on
recent results showing that the appropriate abstract representation of a
problem is specified by the agent's skills. We describe how such a hierarchy
can be used for fast planning, and illustrate the construction of an
appropriate hierarchy for the Taxi domain.
