  Discriminative learning machines often need a large set of labeled samples
for training. Active learning (AL) settings assume that the learner has the
freedom to ask an oracle to label its desired samples. Traditional AL
algorithms heuristically choose query samples about which the current learner
is uncertain. This strategy does not make good use of the structure of the
dataset at hand and is prone to be misguided by outliers. To alleviate this
problem, we propose to distill the structural information into a probabilistic
generative model which acts as a \emph{teacher} in our model. The active
\emph{learner} uses this information effectively at each cycle of active
learning. The proposed method is generic and does not depend on the type of
learner and teacher. We then suggest a query criterion for active learning that
is aware of distribution of data and is more robust against outliers. Our
method can be combined readily with several other query criteria for active
learning. We provide the formulation and empirically show our idea via toy and
real examples.
