  Active learning is designed to minimize annotation efforts by prioritizing
instances that most enhance learning. However, many active learning strategies
struggle with a 'cold start' problem, needing substantial initial data to be
effective. This limitation often reduces their utility for pre-trained models,
which already perform well in few-shot scenarios. To address this, we introduce
ActiveLLM, a novel active learning approach that leverages large language
models such as GPT-4, Llama 3, and Mistral Large for selecting instances. We
demonstrate that ActiveLLM significantly enhances the classification
performance of BERT classifiers in few-shot scenarios, outperforming both
traditional active learning methods and the few-shot learning method SetFit.
Additionally, ActiveLLM can be extended to non-few-shot scenarios, allowing for
iterative selections. In this way, ActiveLLM can even help other active
learning strategies to overcome their cold start problem. Our results suggest
that ActiveLLM offers a promising solution for improving model performance
across various learning setups.
