  Unsupervised skill learning aims to learn a rich repertoire of behaviors
without external supervision, providing artificial agents with the ability to
control and influence the environment. However, without appropriate knowledge
and exploration, skills may provide control only over a restricted area of the
environment, limiting their applicability. Furthermore, it is unclear how to
leverage the learned skill behaviors for adapting to downstream tasks in a
data-efficient manner. We present Choreographer, a model-based agent that
exploits its world model to learn and adapt skills in imagination. Our method
decouples the exploration and skill learning processes, being able to discover
skills in the latent state space of the model. During adaptation, the agent
uses a meta-controller to evaluate and adapt the learned skills efficiently by
deploying them in parallel in imagination. Choreographer is able to learn
skills both from offline data, and by collecting data simultaneously with an
exploration policy. The skills can be used to effectively adapt to downstream
tasks, as we show in the URL benchmark, where we outperform previous approaches
from both pixels and states inputs. The learned skills also explore the
environment thoroughly, finding sparse rewards more frequently, as shown in
goal-reaching tasks from the DMC Suite and Meta-World. Website and code:
https://skillchoreographer.github.io/
