  Spectral super-resolution (SSR) aims at generating a hyperspectral image
(HSI) from a given RGB image. Recently, a promising direction for SSR is to
learn a complicated mapping function from the RGB image to the HSI counterpart
using a deep convolutional neural network. This essentially involves mapping
the RGB context within a size-specific receptive field centered at each pixel
to its spectrum in the HSI. The focus thereon is to appropriately determine the
receptive field size and establish the mapping function from RGB context to the
corresponding spectrum. Due to their differences in category or spatial
position, pixels in HSIs often require different-sized receptive fields and
distinct mapping functions. However, few efforts have been invested to
explicitly exploit this prior.
  To address this problem, we propose a pixel-aware deep function-mixture
network for SSR, which is composed of a new class of modules, termed
function-mixture (FM) blocks. Each FM block is equipped with some basis
functions, i.e., parallel subnets of different-sized receptive fields. Besides,
it incorporates an extra subnet as a mixing function to generate pixel-wise
weights, and then linearly mixes the outputs of all basis functions with those
generated weights. This enables us to pixel-wisely determine the receptive
field size and the mapping function. Moreover, we stack several such FM blocks
to further increase the flexibility of the network in learning the pixel-wise
mapping. To encourage feature reuse, intermediate features generated by the FM
blocks are fused in late stage, which proves to be effective for boosting the
SSR performance. Experimental results on three benchmark HSI datasets
demonstrate the superiority of the proposed method.
