  Stochastic encoders have been used in rate-distortion theory and neural
compression because they can be easier to handle. However, in performance
comparisons with deterministic encoders they often do worse, suggesting that
noise in the encoding process may generally be a bad idea. It is poorly
understood if and when stochastic encoders do better than deterministic
encoders. In this paper we provide one illustrative example which shows that
stochastic encoders can significantly outperform the best deterministic
encoders. Our toy example suggests that stochastic encoders may be particularly
useful in the regime of "perfect perceptual quality".
