  Policy robustness in Reinforcement Learning may not be desirable at any cost:
the alterations caused by robustness requirements from otherwise optimal
policies should be explainable, quantifiable and formally verifiable. In this
work we study how policies can be maximally robust to arbitrary observational
noise by analysing how they are altered by this noise through a stochastic
linear operator interpretation of the disturbances, and establish connections
between robustness and properties of the noise kernel and of the underlying
MDPs. Then, we construct sufficient conditions for policy robustness, and
propose a robustness-inducing scheme, applicable to any policy gradient
algorithm, that formally trades off expected policy utility for robustness
through lexicographic optimisation, while preserving convergence and
sub-optimality in the policy synthesis.
