  Continual learning (CL) methods designed for natural image classification
often fail to reach basic quality standards for medical image segmentation.
Atlas-based segmentation, a well-established approach in medical imaging,
incorporates domain knowledge on the region of interest, leading to
semantically coherent predictions. This is especially promising for CL, as it
allows us to leverage structural information and strike an optimal balance
between model rigidity and plasticity over time. When combined with
privacy-preserving prototypes, this process offers the advantages of
rehearsal-based CL without compromising patient privacy. We propose Atlas
Replay, an atlas-based segmentation approach that uses prototypes to generate
high-quality segmentation masks through image registration that maintain
consistency even as the training distribution changes. We explore how our
proposed method performs compared to state-of-the-art CL methods in terms of
knowledge transferability across seven publicly available prostate segmentation
datasets. Prostate segmentation plays a vital role in diagnosing prostate
cancer, however, it poses challenges due to substantial anatomical variations,
benign structural differences in older age groups, and fluctuating acquisition
parameters. Our results show that Atlas Replay is both robust and generalizes
well to yet-unseen domains while being able to maintain knowledge, unlike
end-to-end segmentation methods. Our code base is available under
https://github.com/MECLabTUDA/Atlas-Replay.
