  This paper proposes a new glass segmentation method utilizing paired RGB and
thermal images. Due to the large difference between the transmission property
of visible light and that of the thermal energy through the glass where most
glass is transparent to the visible light but opaque to thermal energy, glass
regions of a scene are made more distinguishable with a pair of RGB and thermal
images than solely with an RGB image. To exploit such a unique property, we
propose a neural network architecture that effectively combines an RGB-thermal
image pair with a new multi-modal fusion module based on attention, and
integrate CNN and transformer to extract local features and non-local
dependencies, respectively. As well, we have collected a new dataset containing
5551 RGB-thermal image pairs with ground-truth segmentation annotations. The
qualitative and quantitative evaluations demonstrate the effectiveness of the
proposed approach on fusing RGB and thermal data for glass segmentation. Our
code and data are available at
https://github.com/Dong-Huo/RGB-T-Glass-Segmentation.
