  Unlike humans, who are capable of continual learning over their lifetimes,
artificial neural networks have long been known to suffer from a phenomenon
known as catastrophic forgetting, whereby new learning can lead to abrupt
erasure of previously acquired knowledge. Whereas in a neural network the
parameters are typically modelled as scalar values, an individual synapse in
the brain comprises a complex network of interacting biochemical components
that evolve at different timescales. In this paper, we show that by equipping
tabular and deep reinforcement learning agents with a synaptic model that
incorporates this biological complexity (Benna & Fusi, 2016), catastrophic
forgetting can be mitigated at multiple timescales. In particular, we find that
as well as enabling continual learning across sequential training of two simple
tasks, it can also be used to overcome within-task forgetting by reducing the
need for an experience replay database.
