  Depth reconstruction and hyperspectral reflectance reconstruction are two
active research topics in computer vision and image processing. Conventionally,
these two topics have been studied separately using independent imaging setups
and there is no existing method which can acquire depth and spectral
reflectance simultaneously in one shot without using special hardware. In this
paper, we propose a novel single-shot hyperspectral-depth reconstruction method
using an off-the-shelf RGB camera and projector. Our method is based on a
single color-dot projection, which simultaneously acts as structured light for
depth reconstruction and spatially-varying color illuminations for
hyperspectral reflectance reconstruction. To jointly reconstruct the depth and
the hyperspectral reflectance from a single color-dot image, we propose a novel
end-to-end network architecture that effectively incorporates a geometric
color-dot pattern loss and a photometric hyperspectral reflectance loss.
Through the experiments, we demonstrate that our hyperspectral-depth
reconstruction method outperforms the combination of an existing
state-of-the-art single-shot hyperspectral reflectance reconstruction method
and depth reconstruction method.
