  Recent studies in lossy compression show that distortion and perceptual
quality are at odds with each other, which put forward the tradeoff between
distortion and perception (D-P). Intuitively, to attain different perceptual
quality, different decoders have to be trained. In this paper, we present a
nontrivial finding that only two decoders are sufficient for optimally
achieving arbitrary (an infinite number of different) D-P tradeoff. We prove
that arbitrary points of the D-P tradeoff bound can be achieved by a simple
linear interpolation between the outputs of a minimum MSE decoder and a
specifically constructed perfect perceptual decoder. Meanwhile, the perceptual
quality (in terms of the squared Wasserstein-2 distance metric) can be
quantitatively controlled by the interpolation factor. Furthermore, to
construct a perfect perceptual decoder, we propose two theoretically optimal
training frameworks. The new frameworks are different from the
distortion-plus-adversarial loss based heuristic framework widely used in
existing methods, which are not only theoretically optimal but also can yield
state-of-the-art performance in practical perceptual decoding. Finally, we
validate our theoretical finding and demonstrate the superiority of our
frameworks via experiments. Code is available at:
https://github.com/ZeyuYan/Controllable-Perceptual-Compression
