  Hyperspectral image segmentation is crucial for many fields such as
agriculture, remote sensing, biomedical imaging, battlefield sensing and
astronomy. However, the challenge of hyper and multi spectral imaging is its
large data footprint. We propose both a novel camera design and a vision
transformer-based (ViT) algorithm that alleviate both the captured data
footprint and the computational load for hyperspectral segmentation. Our camera
is able to adaptively sample image regions or patches at different resolutions,
instead of capturing the entire hyperspectral cube at one high resolution. Our
segmentation algorithm works in concert with the camera, applying ViT-based
segmentation only to adaptively selected patches. We show results both in
simulation and on a real hardware platform demonstrating both accurate
segmentation results and reduced computational burden.
