  Deep models are highly susceptible to adversarial attacks. Such attacks are
carefully crafted imperceptible noises that can fool the network and can cause
severe consequences when deployed. To encounter them, the model requires
training data for adversarial training or explicit regularization-based
techniques. However, privacy has become an important concern, restricting
access to only trained models but not the training data (e.g. biometric data).
Also, data curation is expensive and companies may have proprietary rights over
it. To handle such situations, we propose a completely novel problem of
'test-time adversarial defense in absence of training data and even their
statistics'. We solve it in two stages: a) detection and b) correction of
adversarial samples. Our adversarial sample detection framework is initially
trained on arbitrary data and is subsequently adapted to the unlabelled test
data through unsupervised domain adaptation. We further correct the predictions
on detected adversarial samples by transforming them in Fourier domain and
obtaining their low frequency component at our proposed suitable radius for
model prediction. We demonstrate the efficacy of our proposed technique via
extensive experiments against several adversarial attacks and for different
model architectures and datasets. For a non-robust Resnet-18 model pre-trained
on CIFAR-10, our detection method correctly identifies 91.42% adversaries.
Also, we significantly improve the adversarial accuracy from 0% to 37.37% with
a minimal drop of 0.02% in clean accuracy on state-of-the-art 'Auto Attack'
without having to retrain the model.
