  Recent Active Learning (AL) approaches in Natural Language Processing (NLP)
proposed using off-the-shelf pretrained language models (LMs). In this paper,
we argue that these LMs are not adapted effectively to the downstream task
during AL and we explore ways to address this issue. We suggest to first adapt
the pretrained LM to the target task by continuing training with all the
available unlabeled data and then use it for AL. We also propose a simple yet
effective fine-tuning method to ensure that the adapted LM is properly trained
in both low and high resource scenarios during AL. Our experiments demonstrate
that our approach provides substantial data efficiency improvements compared to
the standard fine-tuning approach, suggesting that a poor training strategy can
be catastrophic for AL.
