  The advancements in generative modeling, particularly the advent of diffusion
models, have sparked a fundamental question: how can these models be
effectively used for discriminative tasks? In this work, we find that
generative models can be great test-time adapters for discriminative models.
Our method, Diffusion-TTA, adapts pre-trained discriminative models such as
image classifiers, segmenters and depth predictors, to each unlabelled example
in the test set using generative feedback from a diffusion model. We achieve
this by modulating the conditioning of the diffusion model using the output of
the discriminative model. We then maximize the image likelihood objective by
backpropagating the gradients to discriminative model's parameters. We show
Diffusion-TTA significantly enhances the accuracy of various large-scale
pre-trained discriminative models, such as, ImageNet classifiers, CLIP models,
image pixel labellers and image depth predictors. Diffusion-TTA outperforms
existing test-time adaptation methods, including TTT-MAE and TENT, and
particularly shines in online adaptation setups, where the discriminative model
is continually adapted to each example in the test set. We provide access to
code, results, and visualizations on our website:
https://diffusion-tta.github.io/.
