  We introduce a new continual (or lifelong) learning algorithm called LDA-CP&S
that performs segmentation tasks without undergoing catastrophic forgetting.
The method is applied to two different surface defect segmentation problems
that are learned incrementally, i.e. providing data about one type of defect at
a time, while still being capable of predicting every defect that was seen
previously. Our method creates a defect-related subnetwork for each defect type
via iterative pruning and trains a classifier based on linear discriminant
analysis (LDA). At the inference stage, we first predict the defect type with
LDA and then predict the surface defects using the selected subnetwork. We
compare our method with other continual learning methods showing a significant
improvement -- mean Intersection over Union better by a factor of two when
compared to existing methods on both datasets. Importantly, our approach shows
comparable results with joint training when all the training data (all defects)
are seen simultaneously
