  The spectacular successes of recurrent neural network models where key
parameters are adjusted via backpropagation-based gradient descent have
inspired much thought as to how biological neuronal networks might solve the
corresponding synaptic credit assignment problem. There is so far little
agreement, however, as to how biological networks could implement the necessary
backpropagation through time, given widely recognized constraints of biological
synaptic network signaling architectures. Here, we propose that extra-synaptic
diffusion of local neuromodulators such as neuropeptides may afford an
effective mode of backpropagation lying within the bounds of biological
plausibility. Going beyond existing temporal truncation-based gradient
approximations, our approximate gradient-based update rule, ModProp, propagates
credit information through arbitrary time steps. ModProp suggests that
modulatory signals can act on receiving cells by convolving their eligibility
traces via causal, time-invariant and synapse-type-specific filter taps. Our
mathematical analysis of ModProp learning, together with simulation results on
benchmark temporal tasks, demonstrate the advantage of ModProp over existing
biologically-plausible temporal credit assignment rules. These results suggest
a potential neuronal mechanism for signaling credit information related to
recurrent interactions over a longer time horizon. Finally, we derive an
in-silico implementation of ModProp that could serve as a low-complexity and
causal alternative to backpropagation through time.
