  One stream of reinforcement learning research is exploring biologically
plausible models and algorithms to simulate biological intelligence and fit
neuromorphic hardware. Among them, reward-modulated spike-timing-dependent
plasticity (R-STDP) is a recent branch with good potential in energy
efficiency. However, current R-STDP methods rely on heuristic designs of local
learning rules, thus requiring task-specific expert knowledge. In this paper,
we consider a spiking recurrent winner-take-all network, and propose a new
R-STDP method, spiking variational policy gradient (SVPG), whose local learning
rules are derived from the global policy gradient and thus eliminate the need
for heuristic designs. In experiments of MNIST classification and Gym
InvertedPendulum, our SVPG achieves good training performance, and also
presents better robustness to various kinds of noises than conventional
methods.
