  We propose a new iterative segmentation model which can be accurately learned
from a small dataset. A common approach is to train a model to directly segment
an image, requiring a large collection of manually annotated images to capture
the anatomical variability in a cohort. In contrast, we develop a segmentation
model that recursively evolves a segmentation in several steps, and implement
it as a recurrent neural network. We learn model parameters by optimizing the
interme- diate steps of the evolution in addition to the final segmentation. To
this end, we train our segmentation propagation model by presenting incom-
plete and/or inaccurate input segmentations paired with a recommended next
step. Our work aims to alleviate challenges in segmenting heart structures from
cardiac MRI for patients with congenital heart disease (CHD), which encompasses
a range of morphological deformations and topological changes. We demonstrate
the advantages of this approach on a dataset of 20 images from CHD patients,
learning a model that accurately segments individual heart chambers and great
vessels. Com- pared to direct segmentation, the iterative method yields more
accurate segmentation for patients with the most severe CHD malformations.
