  The demands on visual recognition systems do not end with the complexity
offered by current large-scale image datasets, such as ImageNet. In
consequence, we need curious and continuously learning algorithms that actively
acquire knowledge about semantic concepts which are present in available
unlabeled data. As a step towards this goal, we show how to perform continuous
active learning and exploration, where an algorithm actively selects relevant
batches of unlabeled examples for annotation. These examples could either
belong to already known or to yet undiscovered classes. Our algorithm is based
on a new generalization of the Expected Model Output Change principle for deep
architectures and is especially tailored to deep neural networks. Furthermore,
we show easy-to-implement approximations that yield efficient techniques for
active selection. Empirical experiments show that our method outperforms
currently used heuristics.
