  The "fire together, wire together" Hebbian model is a central principle for
learning in neuroscience, but surprisingly, it has found limited applicability
in modern machine learning. In this paper, we take a first step towards
bridging this gap, by developing flavors of competitive Hebbian learning which
produce sparse, distributed neural codes using online adaptation with minimal
tuning. We propose an unsupervised algorithm, termed Adaptive Hebbian Learning
(AHL). We illustrate the distributed nature of the learned representations via
output entropy computations for synthetic data, and demonstrate superior
performance, compared to standard alternatives such as autoencoders, in
training a deep convolutional net on standard image datasets.
