  We initiate the study of fairness in reinforcement learning, where the
actions of a learning algorithm may affect its environment and future rewards.
Our fairness constraint requires that an algorithm never prefers one action
over another if the long-term (discounted) reward of choosing the latter action
is higher. Our first result is negative: despite the fact that fairness is
consistent with the optimal policy, any learning algorithm satisfying fairness
must take time exponential in the number of states to achieve non-trivial
approximation to the optimal policy. We then provide a provably fair polynomial
time algorithm under an approximate notion of fairness, thus establishing an
exponential gap between exact and approximate fairness
