  Deep learning has been widely accepted as a promising solution for medical
image segmentation, given a sufficiently large representative dataset of images
with corresponding annotations. With ever increasing amounts of annotated
medical datasets, it is infeasible to train a learning method always with all
data from scratch. This is also doomed to hit computational limits, e.g.,
memory or runtime feasible for training. Incremental learning can be a
potential solution, where new information (images or anatomy) is introduced
iteratively. Nevertheless, for the preservation of the collective information,
it is essential to keep some "important" (i.e. representative) images and
annotations from the past, while adding new information. In this paper, we
introduce a framework for applying incremental learning for segmentation and
propose novel methods for selecting representative data therein. We
comparatively evaluate our methods in different scenarios using MR images and
validate the increased learning capacity with using our methods.
