  Like humans, deep networks have been shown to learn better when samples are
organized and introduced in a meaningful order or curriculum. Conventional
curriculum learning schemes introduce samples in their order of difficulty.
This forces models to begin learning from a subset of the available data while
adding the external overhead of evaluating the difficulty of samples. In this
work, we propose Learning with Incremental Labels and Adaptive Compensation
(LILAC), a two-phase method that incrementally increases the number of unique
output labels rather than the difficulty of samples while consistently using
the entire dataset throughout training. In the first phase, Incremental Label
Introduction, we partition data into mutually exclusive subsets, one that
contains a subset of the ground-truth labels and another that contains the
remaining data attached to a pseudo-label. Throughout the training process, we
recursively reveal unseen ground-truth labels in fixed increments until all the
labels are known to the model. In the second phase, Adaptive Compensation, we
optimize the loss function using altered target vectors of previously
misclassified samples. The target vectors of such samples are modified to a
smoother distribution to help models learn better. On evaluating across three
standard image benchmarks, CIFAR-10, CIFAR-100, and STL-10, we show that LILAC
outperforms all comparable baselines. Further, we detail the importance of
pacing the introduction of new labels to a model as well as the impact of using
a smooth target vector.
