  Adapting pre-trained language models (PLMs) for time-series text
classification amidst evolving domain shifts (EDS) is critical for maintaining
accuracy in applications like stance detection. This study benchmarks the
effectiveness of evolving domain adaptation (EDA) strategies, notably
self-training, domain-adversarial training, and domain-adaptive pretraining,
with a focus on an incremental self-training method. Our analysis across
various datasets reveals that this incremental method excels at adapting PLMs
to EDS, outperforming traditional domain adaptation techniques. These findings
highlight the importance of continually updating PLMs to ensure their
effectiveness in real-world applications, paving the way for future research
into PLM robustness against the natural temporal evolution of language.
