  Pursuing realistic results according to human visual perception is the
central concern in the image transformation tasks. Perceptual learning
approaches like perceptual loss are empirically powerful for such tasks but
they usually rely on the pre-trained classification network to provide
features, which are not necessarily optimal in terms of visual perception of
image transformation. In this paper, we argue that, among the features
representation from the pre-trained classification network, only limited
dimensions are related to human visual perception, while others are irrelevant,
although both will affect the final image transformation results. Under such an
assumption, we try to disentangle the perception-relevant dimensions from the
representation through our proposed online contrastive learning. The resulted
network includes the pre-training part and a feature selection layer, followed
by the contrastive learning module, which utilizes the transformed results,
target images, and task-oriented distorted images as the positive, negative,
and anchor samples, respectively. The contrastive learning aims at activating
the perception-relevant dimensions and suppressing the irrelevant ones by using
the triplet loss, so that the original representation can be disentangled for
better perceptual quality. Experiments on various image transformation tasks
demonstrate the superiority of our framework, in terms of human visual
perception, to the existing approaches using pre-trained networks and
empirically designed losses.
