  Reinforcement learning tasks are typically specified as Markov decision
processes. This formalism has been highly successful, though specifications
often couple the dynamics of the environment and the learning objective. This
lack of modularity can complicate generalization of the task specification, as
well as obfuscate connections between different task settings, such as episodic
and continuing. In this work, we introduce the RL task formalism, that provides
a unification through simple constructs including a generalization to
transition-based discounting. Through a series of examples, we demonstrate the
generality and utility of this formalism. Finally, we extend standard learning
constructs, including Bellman operators, and extend some seminal theoretical
results, including approximation errors bounds. Overall, we provide a
well-understood and sound formalism on which to build theoretical results and
simplify algorithm use and development.
