  Multi-modal test-time adaptation (MM-TTA) is proposed to adapt models to an
unlabeled target domain by leveraging the complementary multi-modal inputs in
an online manner. Previous MM-TTA methods for 3D segmentation rely on
predictions of cross-modal information in each input frame, while they ignore
the fact that predictions of geometric neighborhoods within consecutive frames
are highly correlated, leading to unstable predictions across time. To fulfill
this gap, we propose ReLiable Spatial-temporal Voxels (Latte), an MM-TTA method
that leverages reliable cross-modal spatial-temporal correspondences for
multi-modal 3D segmentation. Motivated by the fact that reliable predictions
should be consistent with their spatial-temporal correspondences, Latte
aggregates consecutive frames in a slide window manner and constructs
Spatial-Temopral (ST) voxels to capture temporally local prediction consistency
for each modality. After filtering out ST voxels with high ST entropy, Latte
conducts cross-modal learning for each point and pixel by attending to those
with reliable and consistent predictions among both spatial and temporal
neighborhoods. Experimental results show that Latte achieves state-of-the-art
performance on three different MM-TTA benchmarks compared to previous MM-TTA or
TTA methods. Visit our project site https://sites.google.com/view/eccv24-latte.
