  Currently intelligent diagnosis systems lack the ability of continually
learning to diagnose new diseases once deployed, under the condition of
preserving old disease knowledge. In particular, updating an intelligent
diagnosis system with training data of new diseases would cause catastrophic
forgetting of old disease knowledge. To address the catastrophic forgetting
issue, an Adapter-based Continual Learning framework called ACL is proposed to
help effectively learn a set of new diseases at each round (or task) of
continual learning, without changing the shared feature extractor. The
learnable lightweight task-specific adapter(s) can be flexibly designed (e.g.,
two convolutional layers) and then added to the pretrained and fixed feature
extractor. Together with a specially designed task-specific head which absorbs
all previously learned old diseases as a single "out-of-distribution" category,
task-specific adapter(s) can help the pretrained feature extractor more
effectively extract discriminative features between diseases. In addition, a
simple yet effective fine-tuning is applied to collaboratively fine-tune
multiple task-specific heads such that outputs from different heads are
comparable and consequently the appropriate classifier head can be more
accurately selected during model inference. Extensive empirical evaluations on
three image datasets demonstrate the superior performance of ACL in continual
learning of new diseases. The source code is available at
https://github.com/GiantJun/CL_Pytorch.
