  Efficient exploration for automatic subgoal discovery is a challenging
problem in Hierarchical Reinforcement Learning (HRL). In this paper, we show
that intrinsic motivation learning increases the efficiency of exploration,
leading to successful subgoal discovery. We introduce a model-free subgoal
discovery method based on unsupervised learning over a limited memory of
agent's experiences during intrinsic motivation. Additionally, we offer a
unified approach to learning representations in model-free HRL.
