  Unsupervised Reinforcement Learning (RL) provides a promising paradigm for
learning useful behaviors via reward-free per-training. Existing methods for
unsupervised RL mainly conduct empowerment-driven skill discovery or
entropy-based exploration. However, empowerment often leads to static skills,
and pure exploration only maximizes the state coverage rather than learning
useful behaviors. In this paper, we propose a novel unsupervised RL framework
via an ensemble of skills, where each skill performs partition exploration
based on the state prototypes. Thus, each skill can explore the clustered area
locally, and the ensemble skills maximize the overall state coverage. We adopt
state-distribution constraints for the skill occupancy and the desired cluster
for learning distinguishable skills. Theoretical analysis is provided for the
state entropy and the resulting skill distributions. Based on extensive
experiments on several challenging tasks, we find our method learns
well-explored ensemble skills and achieves superior performance in various
downstream tasks compared to previous methods.
