  Test-time adaptation harnesses test inputs to improve the accuracy of a model
trained on source data when tested on shifted target data. Existing methods
update the source model by (re-)training on each target domain. While
effective, re-training is sensitive to the amount and order of the data and the
hyperparameters for optimization. We instead update the target data, by
projecting all test inputs toward the source domain with a generative diffusion
model. Our diffusion-driven adaptation method, DDA, shares its models for
classification and generation across all domains. Both models are trained on
the source domain, then fixed during testing. We augment diffusion with image
guidance and self-ensembling to automatically decide how much to adapt. Input
adaptation by DDA is more robust than prior model adaptation approaches across
a variety of corruptions, architectures, and data regimes on the ImageNet-C
benchmark. With its input-wise updates, DDA succeeds where model adaptation
degrades on too little data in small batches, dependent data in non-uniform
order, or mixed data with multiple corruptions.
