  In lifelong learning systems based on artificial neural networks, one of the
biggest obstacles is the inability to retain old knowledge as new information
is encountered. This phenomenon is known as catastrophic forgetting. In this
paper, we propose a new kind of connectionist architecture, the Sequential
Neural Coding Network, that is robust to forgetting when learning from streams
of data points and, unlike networks of today, does not learn via the popular
back-propagation of errors. Grounded in the neurocognitive theory of predictive
processing, our model adapts synapses in a biologically-plausible fashion while
another neural system learns to direct and control this cortex-like structure,
mimicking some of the task-executive control functionality of the basal
ganglia. In our experiments, we demonstrate that our self-organizing system
experiences significantly less forgetting compared to standard neural models,
outperforming a swath of previously proposed methods, including rehearsal/data
buffer-based methods, on both standard (SplitMNIST, Split Fashion MNIST, etc.)
and custom benchmarks even though it is trained in a stream-like fashion. Our
work offers evidence that emulating mechanisms in real neuronal systems, e.g.,
local learning, lateral competition, can yield new directions and possibilities
for tackling the grand challenge of lifelong machine learning.
