  Lossy image compression is often limited by the simplicity of the chosen loss
measure. Recent research suggests that generative adversarial networks have the
ability to overcome this limitation and serve as a multi-modal loss, especially
for textures. Together with learned image compression, these two techniques can
be used to great effect when relaxing the commonly employed tight measures of
distortion. However, convolutional neural network based algorithms have a large
computational footprint. Ideally, an existing conventional codec should stay in
place, which would ensure faster adoption and adhering to a balanced
computational envelope.
  As a possible avenue to this goal, in this work, we propose and investigate
how learned image coding can be used as a surrogate to optimize an image for
encoding. The image is altered by a learned filter to optimise for a different
performance measure or a particular task. Extending this idea with a generative
adversarial network, we show how entire textures are replaced by ones that are
less costly to encode but preserve sense of detail.
  Our approach can remodel a conventional codec to adjust for the MS-SSIM
distortion with over 20% rate improvement without any decoding overhead. On
task-aware image compression, we perform favourably against a similar but
codec-specific approach.
