  The Backprop algorithm for learning in neural networks utilizes two
mechanisms: first, stochastic gradient descent and second, initialization with
small random weights, where the latter is essential to the effectiveness of the
former. We show that in continual learning setups, Backprop performs well
initially, but over time its performance degrades. Stochastic gradient descent
alone is insufficient to learn continually; the initial randomness enables only
initial learning but not continual learning. To the best of our knowledge, ours
is the first result showing this degradation in Backprop's ability to learn. To
address this degradation in Backprop's plasticity, we propose an algorithm that
continually injects random features alongside gradient descent using a new
generate-and-test process. We call this the \textit{Continual Backprop}
algorithm. We show that, unlike Backprop, Continual Backprop is able to
continually adapt in both supervised and reinforcement learning (RL) problems.
Continual Backprop has the same computational complexity as Backprop and can be
seen as a natural extension of Backprop for continual learning.
