  Data-fusion networks have shown significant promise for RGB-thermal scene
parsing. However, the majority of existing studies have relied on symmetric
duplex encoders for heterogeneous feature extraction and fusion, paying
inadequate attention to the inherent differences between RGB and thermal
modalities. Recent progress in vision foundation models (VFMs) trained through
self-supervision on vast amounts of unlabeled data has proven their ability to
extract informative, general-purpose features. However, this potential has yet
to be fully leveraged in the domain. In this study, we take one step toward
this new research area by exploring a feasible strategy to fully exploit VFM
features for RGB-thermal scene parsing. Specifically, we delve deeper into the
unique characteristics of RGB and thermal modalities, thereby designing a
hybrid, asymmetric encoder that incorporates both a VFM and a convolutional
neural network. This design allows for more effective extraction of
complementary heterogeneous features, which are subsequently fused in a
dual-path, progressive manner. Moreover, we introduce an auxiliary task to
further enrich the local semantics of the fused features, thereby improving the
overall performance of RGB-thermal scene parsing. Our proposed HAPNet, equipped
with all these components, demonstrates superior performance compared to all
other state-of-the-art RGB-thermal scene parsing networks, achieving top ranks
across three widely used public RGB-thermal scene parsing datasets. We believe
this new paradigm has opened up new opportunities for future developments in
data-fusion scene parsing approaches.
