  Continual Test-Time Adaptation (CTTA) generalizes conventional Test-Time
Adaptation (TTA) by assuming that the target domain is dynamic over time rather
than stationary. In this paper, we explore Multi-Modal Continual Test-Time
Adaptation (MM-CTTA) as a new extension of CTTA for 3D semantic segmentation.
The key to MM-CTTA is to adaptively attend to the reliable modality while
avoiding catastrophic forgetting during continual domain shifts, which is out
of the capability of previous TTA or CTTA methods. To fulfill this gap, we
propose an MM-CTTA method called Continual Cross-Modal Adaptive Clustering
(CoMAC) that addresses this task from two perspectives. On one hand, we propose
an adaptive dual-stage mechanism to generate reliable cross-modal predictions
by attending to the reliable modality based on the class-wise feature-centroid
distance in the latent space. On the other hand, to perform test-time
adaptation without catastrophic forgetting, we design class-wise momentum
queues that capture confident target features for adaptation while
stochastically restoring pseudo-source features to revisit source knowledge. We
further introduce two new benchmarks to facilitate the exploration of MM-CTTA
in the future. Our experimental results show that our method achieves
state-of-the-art performance on both benchmarks.
