  Existing quality enhancement methods for compressed images focus on aligning
the enhancement domain with the raw domain to yield realistic images. However,
these methods exhibit a pervasive enhancement bias towards the compression
domain, inadvertently regarding it as more realistic than the raw domain. This
bias makes enhanced images closely resemble their compressed counterparts, thus
degrading their perceptual quality. In this paper, we propose a simple yet
effective method to mitigate this bias and enhance the quality of compressed
images. Our method employs a conditional discriminator with the compressed
image as a key condition, and then incorporates a domain-divergence
regularization to actively distance the enhancement domain from the compression
domain. Through this dual strategy, our method enables the discrimination
against the compression domain, and brings the enhancement domain closer to the
raw domain. Comprehensive quality evaluations confirm the superiority of our
method over other state-of-the-art methods without incurring inference
overheads.
