  Vision-Language Models (VLMs) such as CLIP have yielded unprecedented
performance for zero-shot image classification, yet their generalization
capability may still be seriously challenged when confronted to domain shifts.
In response, we present Weight Average Test-Time Adaptation (WATT) of CLIP, a
pioneering approach facilitating full test-time adaptation (TTA) of this VLM.
Our method employs a diverse set of templates for text prompts, augmenting the
existing framework of CLIP. Predictions are utilized as pseudo labels for model
updates, followed by weight averaging to consolidate the learned information
globally. Furthermore, we introduce a text ensemble strategy, enhancing overall
test performance by aggregating diverse textual cues. Our findings underscore
the efficacy of WATT in enhancing performance across diverse datasets,
including CIFAR-10-C, CIFAR-10.1, CIFAR-100-C, VisDA-C, and several other
challenging datasets, effectively covering a wide range of domain shifts.
Notably, these enhancements are achieved without necessitating additional model
transformations or trainable modules. Moreover, compared to other Test-Time
Adaptation methods, our approach can operate effectively with just a single
image. Highlighting the potential of innovative test-time strategies, this
research emphasizes their role in fortifying the adaptability of VLMs. The
implementation is available at:
\url{https://github.com/Mehrdad-Noori/WATT.git}.
