  It is generally assumed that the brain uses something akin to sparse
distributed representations. These representations, however, are
high-dimensional and consequently they affect classification performance of
traditional Machine Learning models due to "the curse of dimensionality". In
tasks for which there is a vast amount of labeled data, Deep Networks seem to
solve this issue with many layers and a non-Hebbian backpropagation algorithm.
The brain, however, seems to be able to solve the problem with few layers. In
this work, we hypothesize that this happens by using Hebbian learning.
Actually, the Hebbian-like learning rule of Restricted Boltzmann Machines
learns the input patterns asymmetrically. It exclusively learns the correlation
between non-zero values and ignores the zeros, which represent the vast
majority of the input dimensionality. By ignoring the zeros "the curse of
dimensionality" problem can be avoided. To test our hypothesis, we generated
several sparse datasets and compared the performance of a Restricted Boltzmann
Machine classifier with some Backprop-trained networks. The experiments using
these codes confirm our initial intuition as the Restricted Boltzmann Machine
shows a good generalization performance, while the Neural Networks trained with
the backpropagation algorithm overfit the training data.
