  In order to mimic the human ability of continual acquisition and transfer of
knowledge across various tasks, a learning system needs the capability for
continual learning, effectively utilizing the previously acquired skills. As
such, the key challenge is to transfer and generalize the knowledge learned
from one task to other tasks, avoiding forgetting and interference of previous
knowledge and improving the overall performance. In this paper, within the
continual learning paradigm, we introduce a method that effectively forgets the
less useful data samples continuously and allows beneficial information to be
kept for training of the subsequent tasks, in an online manner. The method uses
statistical leverage score information to measure the importance of the data
samples in every task and adopts frequent directions approach to enable a
continual or life-long learning property. This effectively maintains a constant
training size across all tasks. We first provide mathematical intuition for the
method and then demonstrate its effectiveness in avoiding catastrophic
forgetting and computational efficiency on continual learning of classification
tasks when compared with the existing state-of-the-art techniques.
