  We propose a formal mathematical model for sparse representations and active
dendrites in neocortex. Our model is inspired by recent experimental findings
on active dendritic processing and NMDA spikes in pyramidal neurons. These
experimental and modeling studies suggest that the basic unit of pattern memory
in the neocortex is instantiated by small clusters of synapses operated on by
localized non-linear dendritic processes. We derive a number of scaling laws
that characterize the accuracy of such dendrites in detecting activation
patterns in a neuronal population under adverse conditions. We introduce the
union property which shows that synapses for multiple patterns can be randomly
mixed together within a segment and still lead to highly accurate recognition.
We describe simulation results that provide further insight into sparse
representations as well as two primary results. First we show that pattern
recognition by a neuron with active dendrites can be extremely accurate and
robust with high dimensional sparse inputs even when using a tiny number of
synapses to recognize large patterns. Second, equations representing
recognition accuracy of a dendrite predict optimal NMDA spiking thresholds
under a generous set of assumptions. The prediction tightly matches NMDA
spiking thresholds measured in the literature. Our model matches many of the
known properties of pyramidal neurons. As such the theory provides a
mathematical framework for understanding the benefits and limits of sparse
representations in cortical networks.
