  With the increasing availability of depth sensors, multimodal frameworks that
combine color information with depth data are gaining interest. However, ground
truth data for semantic segmentation is burdensome to provide, thus making
domain adaptation a significant research area. Yet most domain adaptation
methods are not able to effectively handle multimodal data. Specifically, we
address the challenging source-free domain adaptation setting where the
adaptation is performed without reusing source data. We propose MISFIT:
MultImodal Source-Free Information fusion Transformer, a depth-aware framework
which injects depth data into a segmentation module based on vision
transformers at multiple stages, namely at the input, feature and output
levels. Color and depth style transfer helps early-stage domain alignment while
re-wiring self-attention between modalities creates mixed features, allowing
the extraction of better semantic content. Furthermore, a depth-based entropy
minimization strategy is also proposed to adaptively weight regions at
different distances. Our framework, which is also the first approach using
RGB-D vision transformers for source-free semantic segmentation, shows
noticeable performance improvements with respect to standard strategies.
