  Image super-resolution is important in many fields, such as surveillance and
remote sensing. However, infrared (IR) images normally have low resolution
since the optical equipment is relatively expensive. Recently, deep learning
methods have dominated image super-resolution and achieved remarkable
performance on visible images; however, IR images have received less attention.
IR images have fewer patterns, and hence, it is difficult for deep neural
networks (DNNs) to learn diverse features from IR images. In this paper, we
present a framework that employs heterogeneous convolution and adversarial
training, namely, heterogeneous kernel-based super-resolution Wasserstein GAN
(HetSRWGAN), for IR image super-resolution. The HetSRWGAN algorithm is a
lightweight GAN architecture that applies a plug-and-play heterogeneous
kernel-based residual block. Moreover, a novel loss function that employs image
gradients is adopted, which can be applied to an arbitrary model. The proposed
HetSRWGAN achieves consistently better performance in both qualitative and
quantitative evaluations. According to the experimental results, the whole
training process is more stable.
