  Deep Neural Networks (DNNs) have two key deficiencies, their dependence on
high precision computing and their inability to perform sequential learning,
that is, when a DNN is trained on a first task and the same DNN is trained on
the next task it forgets the first task. This phenomenon of forgetting previous
tasks is also referred to as catastrophic forgetting. On the other hand a
mammalian brain outperforms DNNs in terms of energy efficiency and the ability
to learn sequentially without catastrophically forgetting. Here, we use
bio-inspired Spike Timing Dependent Plasticity (STDP)in the feature extraction
layers of the network with instantaneous neurons to extract meaningful
features. In the classification sections of the network we use a modified
synaptic intelligence that we refer to as cost per synapse metric as a
regularizer to immunize the network against catastrophic forgetting in a
Single-Incremental-Task scenario (SIT). In this study, we use MNIST handwritten
digits dataset that was divided into five sub-tasks.
