  This paper proposes a novel batch normalization strategy for test-time
adaptation. Recent test-time adaptation methods heavily rely on the modified
batch normalization, i.e., transductive batch normalization (TBN), which
calculates the mean and the variance from the current test batch rather than
using the running mean and variance obtained from the source data, i.e.,
conventional batch normalization (CBN). Adopting TBN that employs test batch
statistics mitigates the performance degradation caused by the domain shift.
However, re-estimating normalization statistics using test data depends on
impractical assumptions that a test batch should be large enough and be drawn
from i.i.d. stream, and we observed that the previous methods with TBN show
critical performance drop without the assumptions. In this paper, we identify
that CBN and TBN are in a trade-off relationship and present a new test-time
normalization (TTN) method that interpolates the statistics by adjusting the
importance between CBN and TBN according to the domain-shift sensitivity of
each BN layer. Our proposed TTN improves model robustness to shifted domains
across a wide range of batch sizes and in various realistic evaluation
scenarios. TTN is widely applicable to other test-time adaptation methods that
rely on updating model parameters via backpropagation. We demonstrate that
adopting TTN further improves their performance and achieves state-of-the-art
performance in various standard benchmarks.
