  Recent approximations to backpropagation (BP) have mitigated many of BP's
computational inefficiencies and incompatibilities with biology, but important
limitations still remain. Moreover, the approximations significantly decrease
accuracy in benchmarks, suggesting that an entirely different approach may be
more fruitful. Here, grounded on recent theory for Hebbian learning in soft
winner-take-all networks, we present multilayer SoftHebb, i.e. an algorithm
that trains deep neural networks, without any feedback, target, or error
signals. As a result, it achieves efficiency by avoiding weight transport,
non-local plasticity, time-locking of layer updates, iterative equilibria, and
(self-) supervisory or other feedback signals -- which were necessary in other
approaches. Its increased efficiency and biological compatibility do not trade
off accuracy compared to state-of-the-art bio-plausible learning, but rather
improve it. With up to five hidden layers and an added linear classifier,
accuracies on MNIST, CIFAR-10, STL-10, and ImageNet, respectively reach 99.4%,
80.3%, 76.2%, and 27.3%. In conclusion, SoftHebb shows with a radically
different approach from BP that Deep Learning over few layers may be plausible
in the brain and increases the accuracy of bio-plausible machine learning. Code
is available at https://github.com/NeuromorphicComputing/SoftHebb.
