  In this work, we propose a novel complementary learning approach to enhance
test-time adaptation (TTA), which has been proven to exhibit good performance
on testing data with distribution shifts such as corruptions. In test-time
adaptation tasks, information from the source domain is typically unavailable
and the model has to be optimized without supervision for test-time samples.
Hence, usual methods assign labels for unannotated data with the prediction by
a well-trained source model in an unsupervised learning framework. Previous
studies have employed unsupervised objectives, such as the entropy of model
predictions, as optimization targets to effectively learn features for
test-time samples. However, the performance of the model is easily compromised
by the quality of pseudo-labels, since inaccuracies in pseudo-labels introduce
noise to the model. Therefore, we propose to leverage the "less probable
categories" to decrease the risk of incorrect pseudo-labeling. The
complementary label is introduced to designate these categories. We highlight
that the risk function of complementary labels agrees with their Vanilla loss
formula under the conventional true label distribution. Experiments show that
the proposed learning algorithm achieves state-of-the-art performance on
different datasets and experiment settings.
