  We describe an application of Wasserstein distance to Reinforcement Learning.
The Wasserstein distance in question is between the distribution of mappings of
trajectories of a policy into some metric space, and some other fixed
distribution (which may, for example, come from another policy). Different
policies induce different distributions, so given an underlying metric, the
Wasserstein distance quantifies how different policies are. This can be used to
learn multiple polices which are different in terms of such Wasserstein
distances by using a Wasserstein regulariser. Changing the sign of the
regularisation parameter, one can learn a policy for which its trajectory
mapping distribution is attracted to a given fixed distribution.
