  Fine-tuning the pre-trained model with active learning holds promise for
reducing annotation costs. However, this combination introduces significant
computational costs, particularly with the growing scale of pre-trained models.
Recent research has proposed proxy-based active learning, which pre-computes
features to reduce computational costs. Yet, this approach often incurs a
significant loss in active learning performance, which may even outweigh the
computational cost savings. In this paper, we argue the performance drop stems
not only from pre-computed features' inability to distinguish between
categories of labeled samples, resulting in the selection of redundant samples
but also from the tendency to compromise valuable pre-trained information when
fine-tuning with samples selected through the proxy model. To address this
issue, we propose a novel method called aligned selection via proxy to update
pre-computed features while selecting a proper training method to inherit
valuable pre-training information. Extensive experiments validate that our
method significantly improves the total cost of efficient active learning while
maintaining computational efficiency.
