  Domain Adaptation (DA) techniques are important for overcoming the domain
shift between the source domain used for training and the target domain where
testing takes place. However, current DA methods assume that the entire target
domain is available during adaptation, which may not hold in practice. This
paper considers a more realistic scenario, where target data become available
in smaller batches and adaptation on the entire target domain is not feasible.
In our work, we introduce a new, data-constrained DA paradigm where unlabeled
target samples are received in batches and adaptation is performed continually.
We propose a novel source-free method for continual unsupervised domain
adaptation that utilizes a buffer for selective replay of previously seen
samples. In our continual DA framework, we selectively mix samples from
incoming batches with data stored in a buffer using buffer management
strategies and use the combination to incrementally update our model. We
evaluate the classification performance of the continual DA approach with
state-of-the-art DA methods based on the entire target domain. Our results on
three popular DA datasets demonstrate that our method outperforms many existing
state-of-the-art DA methods with access to the entire target domain during
adaptation.
