  Nearly all state-of-the-art deep learning algorithms rely on error
backpropagation, which is generally regarded as biologically implausible. An
alternative way of training an artificial neural network is through treating
each unit in the network as a reinforcement learning agent, and thus the
network is considered as a team of agents. As such, all units can be trained by
REINFORCE, a local learning rule modulated by a global signal that is more
consistent with biologically observed forms of synaptic plasticity. Although
this learning rule follows the gradient of return in expectation, it suffers
from high variance and thus the low speed of learning, rendering it impractical
to train deep networks. We therefore propose a novel algorithm called MAP
propagation to reduce this variance significantly while retaining the local
property of the learning rule. Experiments demonstrated that MAP propagation
could solve common reinforcement learning tasks at a similar speed to
backpropagation when applied to an actor-critic network. Our work thus allows
for the broader application of the teams of agents in deep reinforcement
learning.
