  In this paper, we propose a novel learning scheme called epoch-evolving
Gaussian Process Guided Learning (GPGL), which aims at characterizing the
correlation information between the batch-level distribution and the global
data distribution. Such correlation information is encoded as context labels
and needs renewal every epoch. With the guidance of the context label and
ground truth label, GPGL scheme provides a more efficient optimization through
updating the model parameters with a triangle consistency loss. Furthermore,
our GPGL scheme can be further generalized and naturally applied to the current
deep models, outperforming the existing batch-based state-of-the-art models on
mainstream datasets (CIFAR-10, CIFAR-100, and Tiny-ImageNet) remarkably.
