  In recent years, spiking neural networks (SNNs) have been used in
reinforcement learning (RL) due to their low power consumption and event-driven
features. However, spiking reinforcement learning (SRL), which suffers from
fixed coding methods, still faces the problems of high latency and poor
versatility. In this paper, we use learnable matrix multiplication to encode
and decode spikes, improving the flexibility of the coders and thus reducing
latency. Meanwhile, we train the SNNs using the direct training method and use
two different structures for online and offline RL algorithms, which gives our
model a wider range of applications. Extensive experiments have revealed that
our method achieves optimal performance with ultra-low latency (as low as 0.8%
of other SRL methods) and excellent energy efficiency (up to 5X the DNNs) in
different algorithms and different environments.
