  While the sparse coding principle can successfully model information
processing in sensory neural systems, it remains unclear how learning can be
accomplished under neural architectural constraints. Feasible learning rules
must rely solely on synaptically local information in order to be implemented
on spatially distributed neurons. We describe a neural network with spiking
neurons that can address the aforementioned fundamental challenge and solve the
L1-minimizing dictionary learning problem, representing the first model able to
do so. Our major innovation is to introduce feedback synapses to create a
pathway to turn the seemingly non-local information into local ones. The
resulting network encodes the error signal needed for learning as the change of
network steady states caused by feedback, and operates akin to the classical
stochastic gradient descent method.
