  What is a useful skill hierarchy for an autonomous agent? We propose an
answer based on a graphical representation of how the interaction between an
agent and its environment may unfold. Our approach uses modularity maximisation
as a central organising principle to expose the structure of the interaction
graph at multiple levels of abstraction. The result is a collection of skills
that operate at varying time scales, organised into a hierarchy, where skills
that operate over longer time scales are composed of skills that operate over
shorter time scales. The entire skill hierarchy is generated automatically,
with no human intervention, including the skills themselves (their behaviour,
when they can be called, and when they terminate) as well as the hierarchical
dependency structure between them. In a wide range of environments, this
approach generates skill hierarchies that are intuitively appealing and that
considerably improve the learning performance of the agent.
