  Language features are evolving in real-world social media, resulting in the
deteriorating performance of text classification in dynamics. To address this
challenge, we study temporal adaptation, where models trained on past data are
tested in the future. Most prior work focused on continued pretraining or
knowledge updating, which may compromise their performance on noisy social
media data. To tackle this issue, we reflect feature change via modeling latent
topic evolution and propose a novel model, VIBE: Variational Information
Bottleneck for Evolutions. Concretely, we first employ two Information
Bottleneck (IB) regularizers to distinguish past and future topics. Then, the
distinguished topics work as adaptive features via multi-task training with
timestamp and class label prediction. In adaptive learning, VIBE utilizes
retrieved unlabeled data from online streams created posterior to training data
time. Substantial Twitter experiments on three classification tasks show that
our model, with only 3% of data, significantly outperforms previous
state-of-the-art continued-pretraining methods.
