  Many real-world physical control systems are required to satisfy constraints
upon deployment. Furthermore, real-world systems are often subject to effects
such as non-stationarity, wear-and-tear, uncalibrated sensors and so on. Such
effects effectively perturb the system dynamics and can cause a policy trained
successfully in one domain to perform poorly when deployed to a perturbed
version of the same domain. This can affect a policy's ability to maximize
future rewards as well as the extent to which it satisfies constraints. We
refer to this as constrained model misspecification. We present an algorithm
that mitigates this form of misspecification, and showcase its performance in
multiple simulated Mujoco tasks from the Real World Reinforcement Learning
(RWRL) suite.
