  Reinforcement learning (RL) faces substantial challenges when applied to
real-life problems, primarily stemming from the scarcity of available data due
to limited interactions with the environment. This limitation is exacerbated by
the fact that RL often demands a considerable volume of data for effective
learning. The complexity escalates further when implementing RL in recurrent
spiking networks, where inherent noise introduced by spikes adds a layer of
difficulty. Life-long learning machines must inherently resolve the
plasticity-stability paradox. Striking a balance between acquiring new
knowledge and maintaining stability is crucial for artificial agents. To
address this challenge, we draw inspiration from machine learning technology
and introduce a biologically plausible implementation of proximal policy
optimization, referred to as lf-cs (learning fast changing slow). Our approach
results in two notable advancements: firstly, the capacity to assimilate new
information into a new policy without requiring alterations to the current
policy; and secondly, the capability to replay experiences without experiencing
policy divergence. Furthermore, when contrasted with other experience replay
(ER) techniques, our method demonstrates the added advantage of being
computationally efficient in an online setting. We demonstrate that the
proposed methodology enhances the efficiency of learning, showcasing its
potential impact on neuromorphic and real-world applications.
