  We present label gradient alignment, a novel algorithm for semi-supervised
learning which imputes labels for the unlabeled data and trains on the imputed
labels. We define a semantically meaningful distance metric on the input space
by mapping a point (x, y) to the gradient of the model at (x, y). We then
formulate an optimization problem whose objective is to minimize the distance
between the labeled and the unlabeled data in this space, and we solve it by
gradient descent on the imputed labels. We evaluate label gradient alignment
using the standardized architecture introduced by Oliver et al. (2018) and
demonstrate state-of-the-art accuracy in semi-supervised CIFAR-10
classification.
