  The costly human effort required to prepare the training data of machine
learning (ML) models hinders their practical development and usage in software
engineering (ML4Code), especially for those with limited budgets. Therefore,
efficiently training models of code with less human effort has become an
emergent problem. Active learning is such a technique to address this issue
that allows developers to train a model with reduced data while producing
models with desired performance, which has been well studied in computer vision
and natural language processing domains. Unfortunately, there is no such work
that explores the effectiveness of active learning for code models. In this
paper, we bridge this gap by building the first benchmark to study this
critical problem - active code learning. Specifically, we collect 11
acquisition functions~(which are used for data selection in active learning)
from existing works and adapt them for code-related tasks. Then, we conduct an
empirical study to check whether these acquisition functions maintain
performance for code data. The results demonstrate that feature selection
highly affects active learning and using output vectors to select data is the
best choice. For the code summarization task, active code learning is
ineffective which produces models with over a 29.64\% gap compared to the
expected performance. Furthermore, we explore future directions of active code
learning with an exploratory study. We propose to replace distance calculation
methods with evaluation metrics and find a correlation between these
evaluation-based distance methods and the performance of code models.
