  Traditional color images only depict color intensities in red, green and blue
channels, often making object trackers fail in challenging scenarios, e.g.,
background clutter and rapid changes of target appearance. Alternatively,
material information of targets contained in a large amount of bands of
hyperspectral images (HSI) is more robust to these difficult conditions. In
this paper, we conduct a comprehensive study on how material information can be
utilized to boost object tracking from three aspects: benchmark dataset,
material feature representation and material based tracking. In terms of
benchmark, we construct a dataset of fully-annotated videos, which contain both
hyperspectral and color sequences of the same scene. Material information is
represented by spectral-spatial histogram of multidimensional gradient, which
describes the 3D local spectral-spatial structure in an HSI, and fractional
abundances of constituted material components which encode the underlying
material distribution. These two types of features are embedded into
correlation filters, yielding material based tracking. Experimental results on
the collected benchmark dataset show the potentials and advantages of material
based object tracking.
