  We present a novel extension to the family of Soft Actor-Critic (SAC)
algorithms. We argue that based on the Maximum Entropy Principle, discrete SAC
can be further improved via additional statistical constraints derived from a
surrogate critic policy. Furthermore, our findings suggests that these
constraints provide an added robustness against potential domain shifts, which
are essential for safe deployment of reinforcement learning agents in the
real-world. We provide theoretical analysis and show empirical results on low
data regimes for both in-distribution and out-of-distribution variants of Atari
2600 games.
