  Developed to alleviate prohibitive labeling costs, active learning (AL)
methods aim to reduce label complexity in supervised learning. While recent
work has demonstrated the benefit of using AL in combination with large
pre-trained language models (PLMs), it has often overlooked the practical
challenges that hinder the effectiveness of AL. We address these challenges by
leveraging representation smoothness analysis to ensure AL is feasible, that
is, both effective and practicable. Firstly, we propose an early stopping
technique that does not require a validation set -- often unavailable in
realistic AL conditions -- and observe significant improvements over random
sampling across multiple datasets and AL methods. Further, we find that task
adaptation improves AL, whereas standard short fine-tuning in AL does not
provide improvements over random sampling. Our work demonstrates the usefulness
of representation smoothness analysis for AL and introduces an AL stopping
criterion that reduces label complexity.
