  Acquiring labeled data is challenging in many machine learning applications
with limited budgets. Active learning gives a procedure to select the most
informative data points and improve data efficiency by reducing the cost of
labeling. The info-max learning principle maximizing mutual information such as
BALD has been successful and widely adapted in various active learning
applications. However, this pool-based specific objective inherently introduces
a redundant selection and further requires a high computational cost for batch
selection. In this paper, we design and propose a new uncertainty measure,
Balanced Entropy Acquisition (BalEntAcq), which captures the information
balance between the uncertainty of underlying softmax probability and the label
variable. To do this, we approximate each marginal distribution by Beta
distribution. Beta approximation enables us to formulate BalEntAcq as a ratio
between an augmented entropy and the marginalized joint entropy. The
closed-form expression of BalEntAcq facilitates parallelization by estimating
two parameters in each marginal Beta distribution. BalEntAcq is a purely
standalone measure without requiring any relational computations with other
data points. Nevertheless, BalEntAcq captures a well-diversified selection near
the decision boundary with a margin, unlike other existing uncertainty measures
such as BALD, Entropy, or Mean Standard Deviation (MeanSD). Finally, we
demonstrate that our balanced entropy learning principle with BalEntAcq
consistently outperforms well-known linearly scalable active learning methods,
including a recently proposed PowerBALD, a simple but diversified version of
BALD, by showing experimental results obtained from MNIST, CIFAR-100, SVHN, and
TinyImageNet datasets.
