  Abstraction of Markov Decision Processes is a useful tool for solving complex
problems, as it can ignore unimportant aspects of an environment, simplifying
the process of learning an optimal policy. In this paper, we propose a new
algorithm for finding abstract MDPs in environments with continuous state
spaces. It is based on MDP homomorphisms, a structure-preserving mapping
between MDPs. We demonstrate our algorithm's ability to learn abstractions from
collected experience and show how to reuse the abstractions to guide
exploration in new tasks the agent encounters. Our novel task transfer method
outperforms baselines based on a deep Q-network in the majority of our
experiments. The source code is at https://github.com/ondrejba/aamas_19.
