  Supervised deep learning requires a large amount of training samples with
annotations (e.g. label class for classification task, pixel- or voxel-wised
label map for segmentation tasks), which are expensive and time-consuming to
obtain. During the training of a deep neural network, the annotated samples are
fed into the network in a mini-batch way, where they are often regarded of
equal importance. However, some of the samples may become less informative
during training, as the magnitude of the gradient start to vanish for these
samples. In the meantime, other samples of higher utility or hardness may be
more demanded for the training process to proceed and require more
exploitation. To address the challenges of expensive annotations and loss of
sample informativeness, here we propose a novel training framework which
adaptively selects informative samples that are fed to the training process.
The adaptive selection or sampling is performed based on a hardness-aware
strategy in the latent space constructed by a generative model. To evaluate the
proposed training framework, we perform experiments on three different
datasets, including MNIST and CIFAR-10 for image classification task and a
medical image dataset IVUS for biophysical simulation task. On all three
datasets, the proposed framework outperforms a random sampling method, which
demonstrates the effectiveness of proposed framework.
