  Test-time adaptation (TTA) aims to adapt a trained classifier using online
unlabeled test data only, without any information related to the training
procedure. Most existing TTA methods adapt the trained classifier using the
classifier's prediction on the test data as pseudo-label. However, under
test-time domain shift, accuracy of the pseudo labels cannot be guaranteed, and
thus the TTA methods often encounter performance degradation at the adapted
classifier. To overcome this limitation, we propose a novel test-time
adaptation method, called Test-time Adaptation via Self-Training with nearest
neighbor information (TAST), which is composed of the following procedures: (1)
adds trainable adaptation modules on top of the trained feature extractor; (2)
newly defines a pseudo-label distribution for the test data by using the
nearest neighbor information; (3) trains these modules only a few times during
test time to match the nearest neighbor-based pseudo label distribution and a
prototype-based class distribution for the test data; and (4) predicts the
label of test data using the average predicted class distribution from these
modules. The pseudo-label generation is based on the basic intuition that a
test data and its nearest neighbor in the embedding space are likely to share
the same label under the domain shift. By utilizing multiple randomly
initialized adaptation modules, TAST extracts useful information for the
classification of the test data under the domain shift, using the nearest
neighbor information. TAST showed better performance than the state-of-the-art
TTA methods on two standard benchmark tasks, domain generalization, namely
VLCS, PACS, OfficeHome, and TerraIncognita, and image corruption, particularly
CIFAR-10/100C.
