  Test Time Adaptation (TTA) is a pivotal concept in machine learning, enabling
models to perform well in real-world scenarios, where test data distribution
differs from training. In this work, we propose a novel approach called pseudo
Source guided Target Clustering (pSTarC) addressing the relatively unexplored
area of TTA under real-world domain shifts. This method draws inspiration from
target clustering techniques and exploits the source classifier for generating
pseudo-source samples. The test samples are strategically aligned with these
pseudo-source samples, facilitating their clustering and thereby enhancing TTA
performance. pSTarC operates solely within the fully test-time adaptation
protocol, removing the need for actual source data. Experimental validation on
a variety of domain shift datasets, namely VisDA, Office-Home, DomainNet-126,
CIFAR-100C verifies pSTarC's effectiveness. This method exhibits significant
improvements in prediction accuracy along with efficient computational
requirements. Furthermore, we also demonstrate the universality of the pSTarC
framework by showing its effectiveness for the continuous TTA framework. The
source code for our method is available at https://manogna-s.github.io/pstarc
