  We humans seem to have an innate understanding of the asymmetric progression
of time, which we use to efficiently and safely perceive and manipulate our
environment. Drawing inspiration from that, we address the problem of learning
an arrow of time in a Markov (Decision) Process. We illustrate how a learned
arrow of time can capture meaningful information about the environment, which
in turn can be used to measure reachability, detect side-effects and to obtain
an intrinsic reward signal. We show empirical results on a selection of
discrete and continuous environments, and demonstrate for a class of stochastic
processes that the learned arrow of time agrees reasonably well with a known
notion of an arrow of time given by the celebrated Jordan-Kinderlehrer-Otto
result.
