  The field of Continual Learning (CL) has inspired numerous researchers over
the years, leading to increasingly advanced countermeasures to the issue of
catastrophic forgetting. Most studies have focused on the single-class
scenario, where each example comes with a single label. The recent literature
has successfully tackled such a setting, with impressive results. Differently,
we shift our attention to the multi-label scenario, as we feel it to be more
representative of real-world open problems. In our work, we show that existing
state-of-the-art CL methods fail to achieve satisfactory performance, thus
questioning the real advance claimed in recent years. Therefore, we assess both
old-style and novel strategies and propose, on top of them, an approach called
Selective Class Attention Distillation (SCAD). It relies on a knowledge
transfer technique that seeks to align the representations of the student
network -- which trains continuously and is subject to forgetting -- with the
teacher ones, which is pretrained and kept frozen. Importantly, our method is
able to selectively transfer the relevant information from the teacher to the
student, thereby preventing irrelevant information from harming the student's
performance during online training. To demonstrate the merits of our approach,
we conduct experiments on two different multi-label datasets, showing that our
method outperforms the current state-of-the-art Continual Learning methods. Our
findings highlight the importance of addressing the unique challenges posed by
multi-label environments in the field of Continual Learning. The code of SCAD
is available at https://github.com/aimagelab/SCAD-LOD-2024.
