  Guided image super-resolution (GISR) aims to obtain a high-resolution (HR)
target image by enhancing the spatial resolution of a low-resolution (LR)
target image under the guidance of a HR image. However, previous model-based
methods mainly takes the entire image as a whole, and assume the prior
distribution between the HR target image and the HR guidance image, simply
ignoring many non-local common characteristics between them. To alleviate this
issue, we firstly propose a maximal a posterior (MAP) estimation model for GISR
with two types of prior on the HR target image, i.e., local implicit prior and
global implicit prior. The local implicit prior aims to model the complex
relationship between the HR target image and the HR guidance image from a local
perspective, and the global implicit prior considers the non-local
auto-regression property between the two images from a global perspective.
Secondly, we design a novel alternating optimization algorithm to solve this
model for GISR. The algorithm is in a concise framework that facilitates to be
replicated into commonly used deep network structures. Thirdly, to reduce the
information loss across iterative stages, the persistent memory mechanism is
introduced to augment the information representation by exploiting the Long
short-term memory unit (LSTM) in the image and feature spaces. In this way, a
deep network with certain interpretation and high representation ability is
built. Extensive experimental results validate the superiority of our method on
a variety of GISR tasks, including Pan-sharpening, depth image
super-resolution, and MR image super-resolution.
