  Deep neural networks struggle to continually learn multiple sequential tasks
due to catastrophic forgetting of previously learned tasks. Rehearsal-based
methods which explicitly store previous task samples in the buffer and
interleave them with the current task samples have proven to be the most
effective in mitigating forgetting. However, Experience Replay (ER) does not
perform well under low-buffer regimes and longer task sequences as its
performance is commensurate with the buffer size. Consistency in predictions of
soft-targets can assist ER in preserving information pertaining to previous
tasks better as soft-targets capture the rich similarity structure of the data.
Therefore, we examine the role of consistency regularization in ER framework
under various continual learning scenarios. We also propose to cast consistency
regularization as a self-supervised pretext task thereby enabling the use of a
wide variety of self-supervised learning methods as regularizers. While
simultaneously enhancing model calibration and robustness to natural
corruptions, regularizing consistency in predictions results in lesser
forgetting across all continual learning scenarios. Among the different
families of regularizers, we find that stricter consistency constraints
preserve previous task information in ER better.
