  Intra-operative measurements of tissue shape and multi/ hyperspectral
information have the potential to provide surgical guidance and decision making
support. We report an optical probe based system to combine sparse
hyperspectral measurements and spectrally-encoded structured lighting (SL) for
surface measurements. The system provides informative signals for navigation
with a surgical interface. By rapidly switching between SL and white light (WL)
modes, SL information is combined with structure-from-motion (SfM) from white
light images, based on SURF feature detection and Lucas-Kanade (LK) optical
flow to provide quasi-dense surface shape reconstruction with known scale in
real-time. Furthermore, "super-spectral-resolution" was realized, whereby the
RGB images and sparse hyperspectral data were integrated to recover dense
pixel-level hyperspectral stacks, by using convolutional neural networks to
upscale the wavelength dimension. Validation and demonstration of this system
is reported on ex vivo/in vivo animal/ human experiments.
