  Learning algorithms for Deep Neural Networks are typically based on
supervised end-to-end Stochastic Gradient Descent (SGD) training with error
backpropagation (backprop). Backprop algorithms require a large number of
labelled training samples to achieve high performance. However, in many
realistic applications, even if there is plenty of image samples, very few of
them are labelled, and semi-supervised sample-efficient training strategies
have to be used. Hebbian learning represents a possible approach towards sample
efficient training; however, in current solutions, it does not scale well to
large datasets. In this paper, we present FastHebb, an efficient and scalable
solution for Hebbian learning which achieves higher efficiency by 1) merging
together update computation and aggregation over a batch of inputs, and 2)
leveraging efficient matrix multiplication algorithms on GPU. We validate our
approach on different computer vision benchmarks, in a semi-supervised learning
scenario. FastHebb outperforms previous solutions by up to 50 times in terms of
training speed, and notably, for the first time, we are able to bring Hebbian
algorithms to ImageNet scale.
