  High-resolution (HR) images are usually downscaled to low-resolution (LR)
ones for better display and afterward upscaled back to the original size to
recover details. Recent work in image rescaling formulates downscaling and
upscaling as a unified task and learns a bijective mapping between HR and LR
via invertible networks. However, in real-world applications (e.g., social
media), most images are compressed for transmission. Lossy compression will
lead to irreversible information loss on LR images, hence damaging the inverse
upscaling procedure and degrading the reconstruction accuracy. In this paper,
we propose the Self-Asymmetric Invertible Network (SAIN) for compression-aware
image rescaling. To tackle the distribution shift, we first develop an
end-to-end asymmetric framework with two separate bijective mappings for
high-quality and compressed LR images, respectively. Then, based on empirical
analysis of this framework, we model the distribution of the lost information
(including downscaling and compression) using isotropic Gaussian mixtures and
propose the Enhanced Invertible Block to derive high-quality/compressed LR
images in one forward pass. Besides, we design a set of losses to regularize
the learned LR images and enhance the invertibility. Extensive experiments
demonstrate the consistent improvements of SAIN across various image rescaling
datasets in terms of both quantitative and qualitative evaluation under
standard image compression formats (i.e., JPEG and WebP).
