  Learned image compression is making good progress in recent years. Peak
signal-to-noise ratio (PSNR) and multi-scale structural similarity (MS-SSIM)
are the two most popular evaluation metrics. As different metrics only reflect
certain aspects of human perception, works in this field normally optimize two
models using PSNR and MS-SSIM as loss function separately, which is suboptimal
and makes it difficult to select the model with best visual quality or overall
performance. Towards solving this problem, we propose to Harmonize optimization
metrics in Learned Image Compression (HLIC) using online loss function
adaptation by reinforcement learning. By doing so, we are able to leverage the
advantages of both PSNR and MS-SSIM, achieving better visual quality and higher
VMAF score. To our knowledge, our work is the first to explore automatic loss
function adaptation for harmonizing optimization metrics in low level vision
tasks like learned image compression.
