  To compose a 360 image from a rig with multiple fisheye cameras, a
conventional processing pipeline first performs demosaicking on each fisheye
camera's Bayer-patterned grid, then translates demosaicked pixels from the
camera grid to a rectified image grid---thus performing two image interpolation
steps in sequence. Hence interpolation errors can accumulate, and acquisition
noise in the captured pixels can pollute neighbors in two consecutive
processing stages. In this paper, we propose a joint processing framework that
performs demosaicking and grid-to-grid mapping simultaneously---thus limiting
noise pollution to one interpolation. Specifically, we first obtain a reverse
mapping function from a regular on-grid location in the rectified image to an
irregular off-grid location in the camera's Bayer-patterned image. For each
pair of adjacent pixels in the rectified grid, we estimate its gradient using
the pair's neighboring pixel gradients in three colors in the Bayer-patterned
grid. We construct a similarity graph based on the estimated gradients, and
interpolate pixels in the rectified grid directly via graph Laplacian
regularization (GLR). Experiments show that our joint method outperforms
several competing local methods that execute demosaicking and rectification in
sequence, by up to 0.52 dB in PSNR and 0.086 in SSIM on the publicly available
dataset, and by up to 5.53dB in PSNR and 0.411 in SSIM on the in-house
constructed dataset.
