  Options in reinforcement learning allow agents to hierarchically decompose a
task into subtasks, having the potential to speed up learning and planning.
However, autonomously learning effective sets of options is still a major
challenge in the field. In this paper we focus on the recently introduced idea
of using representation learning methods to guide the option discovery process.
Specifically, we look at eigenoptions, options obtained from representations
that encode diffusive information flow in the environment. We extend the
existing algorithms for eigenoption discovery to settings with stochastic
transitions and in which handcrafted features are not available. We propose an
algorithm that discovers eigenoptions while learning non-linear state
representations from raw pixels. It exploits recent successes in the deep
reinforcement learning literature and the equivalence between proto-value
functions and the successor representation. We use traditional tabular domains
to provide intuition about our approach and Atari 2600 games to demonstrate its
potential.
