  To address the issue of catastrophic forgetting in neural networks, we
propose a novel, simple, and effective solution called neuron-level plasticity
control (NPC). While learning a new task, the proposed method preserves the
knowledge for the previous tasks by controlling the plasticity of the network
at the neuron level. NPC estimates the importance value of each neuron and
consolidates important \textit{neurons} by applying lower learning rates,
rather than restricting individual connection weights to stay close to certain
values. The experimental results on the incremental MNIST (iMNIST) and
incremental CIFAR100 (iCIFAR100) datasets show that neuron-level consolidation
is substantially more effective compared to the connection-level consolidation
approaches.
