  Source-free domain adaptation (SFDA) aims to adapt a source model trained on
a fully-labeled source domain to an unlabeled target domain. Large-data
pre-trained networks are used to initialize source models during source
training, and subsequently discarded. However, source training can cause the
model to overfit to source data distribution and lose applicable target domain
knowledge. We propose to integrate the pre-trained network into the target
adaptation process as it has diversified features important for generalization
and provides an alternate view of features and classification decisions
different from the source model. We propose to distil useful target domain
information through a co-learning strategy to improve target pseudolabel
quality for finetuning the source model. Evaluation on 4 benchmark datasets
show that our proposed strategy improves adaptation performance and can be
successfully integrated with existing SFDA methods. Leveraging modern
pre-trained networks that have stronger representation learning ability in the
co-learning strategy further boosts performance.
