  One important desideratum of lifelong learning aims to discover novel classes
from unlabelled data in a continuous manner. The central challenge is twofold:
discovering and learning novel classes while mitigating the issue of
catastrophic forgetting of established knowledge. To this end, we introduce a
new paradigm called Adaptive Discovering and Merging (ADM) to discover novel
categories adaptively in the incremental stage and integrate novel knowledge
into the model without affecting the original knowledge. To discover novel
classes adaptively, we decouple representation learning and novel class
discovery, and use Triple Comparison (TC) and Probability Regularization (PR)
to constrain the probability discrepancy and diversity for adaptive category
assignment. To merge the learned novel knowledge adaptively, we propose a
hybrid structure with base and novel branches named Adaptive Model Merging
(AMM), which reduces the interference of the novel branch on the old classes to
preserve the previous knowledge, and merges the novel branch to the base model
without performance loss and parameter growth. Extensive experiments on several
datasets show that ADM significantly outperforms existing class-incremental
Novel Class Discovery (class-iNCD) approaches. Moreover, our AMM also benefits
the class-incremental Learning (class-IL) task by alleviating the catastrophic
forgetting problem.
