  Large language models (LLMs) increasingly serve as the backbone for
classifying text associated with distinct domains and simultaneously several
labels (classes). When encountering domain shifts, e.g., classifier of movie
reviews from IMDb to Rotten Tomatoes, adapting such an LLM-based multi-label
classifier is challenging due to incomplete label sets at the target domain and
daunting training overhead. The existing domain adaptation methods address
either image multi-label classifiers or text binary classifiers. In this paper,
we design DALLMi, Domain Adaptation Large Language Model interpolator, a
first-of-its-kind semi-supervised domain adaptation method for text data models
based on LLMs, specifically BERT. The core of DALLMi is the novel variation
loss and MixUp regularization, which jointly leverage the limited positively
labeled and large quantity of unlabeled text and, importantly, their
interpolation from the BERT word embeddings. DALLMi also introduces a
label-balanced sampling strategy to overcome the imbalance between labeled and
unlabeled data. We evaluate DALLMi against the partial-supervised and
unsupervised approach on three datasets under different scenarios of label
availability for the target domain. Our results show that DALLMi achieves
higher mAP than unsupervised and partially-supervised approaches by 19.9% and
52.2%, respectively.
