  We present a novel multiple-source unsupervised model for text classification
under domain shift. Our model exploits the update rates in document
representations to dynamically integrate domain encoders. It also employs a
probabilistic heuristic to infer the error rate in the target domain in order
to pair source classifiers. Our heuristic exploits data transformation cost and
the classifier accuracy in the target feature space. We have used real world
scenarios of Domain Adaptation to evaluate the efficacy of our algorithm. We
also used pretrained multi-layer transformers as the document encoder in the
experiments to demonstrate whether the improvement achieved by domain
adaptation models can be delivered by out-of-the-box language model
pretraining. The experiments testify that our model is the top performing
approach in this setting.
