  While the difficulty of reinforcement learning problems is typically related
to the complexity of their state spaces, Abstraction proposes that solutions
often lie in simpler underlying latent spaces. Prior works have focused on
learning either a continuous or dense abstraction, or require a human to
provide one. Information-dense representations capture features irrelevant for
solving tasks, and continuous spaces can struggle to represent discrete
objects. In this work we automatically learn a sparse discrete abstraction of
the underlying environment. We do so using a simple end-to-end trainable model
based on the successor representation and max-entropy regularization. We
describe an algorithm to apply our model, named Discrete State-Action
Abstraction (DSAA), which computes an action abstraction in the form of
temporally extended actions, i.e., Options, to transition between discrete
abstract states. Empirically, we demonstrate the effects of different
exploration schemes on our resulting abstraction, and show that it is efficient
for solving downstream tasks.
