  This work focuses on tackling the challenging but realistic visual task of
Incremental Few-Shot Learning (IFSL), which requires a model to continually
learn novel classes from only a few examples while not forgetting the base
classes on which it was pre-trained. Our study reveals that the challenges of
IFSL lie in both inter-class separation and novel-class representation. Dur to
intra-class variation, a novel class may implicitly leverage the knowledge from
multiple base classes to construct its feature representation. Hence, simply
reusing the pre-trained embedding space could lead to a scattered feature
distribution and result in category confusion. To address such issues, we
propose a two-step learning strategy referred to as \textbf{Im}planting and
\textbf{Co}mpressing (\textbf{IMCO}), which optimizes both feature space
partition and novel class reconstruction in a systematic manner. Specifically,
in the \textbf{Implanting} step, we propose to mimic the data distribution of
novel classes with the assistance of data-abundant base set, so that a model
could learn semantically-rich features that are beneficial for discriminating
between the base and other unseen classes. In the \textbf{Compressing} step, we
adapt the feature extractor to precisely represent each novel class for
enhancing intra-class compactness, together with a regularized parameter
updating rule for preventing aggressive model updating. Finally, we demonstrate
that IMCO outperforms competing baselines with a significant margin, both in
image classification task and more challenging object detection task.
