  Multimodal image super-resolution (SR) is the reconstruction of a high
resolution image given a low-resolution observation with the aid of another
image modality. While existing deep multimodal models do not incorporate domain
knowledge about image SR, we present a multimodal deep network design that
integrates coupled sparse priors and allows the effective fusion of information
from another modality into the reconstruction process. Our method is inspired
by a novel iterative algorithm for coupled convolutional sparse coding,
resulting in an interpretable network by design. We apply our model to the
super-resolution of near-infrared image guided by RGB images. Experimental
results show that our model outperforms state-of-the-art methods.
