  Planning the optimal time of intervention for pulmonary valve replacement
surgery in patients with the congenital heart disease Tetralogy of Fallot (TOF)
is mainly based on ventricular volume and function according to current
guidelines. Both of these two biomarkers are most reliably assessed by
segmentation of 3D cardiac magnetic resonance (CMR) images. In several grand
challenges in the last years, U-Net architectures have shown impressive results
on the provided data. However, in clinical practice, data sets are more diverse
considering individual pathologies and image properties derived from different
scanner properties. Additionally, specific training data for complex rare
diseases like TOF is scarce.
  For this work, 1) we assessed the accuracy gap when using a publicly
available labelled data set (the Automatic Cardiac Diagnosis Challenge (ACDC)
data set) for training and subsequent applying it to CMR data of TOF patients
and vice versa and 2) whether we can achieve similar results when applying the
model to a more heterogeneous data base.
  Multiple deep learning models were trained with four-fold cross validation.
Afterwards they were evaluated on the respective unseen CMR images from the
other collection. Our results confirm that current deep learning models can
achieve excellent results (left ventricle dice of
$0.951\pm{0.003}$/$0.941\pm{0.007}$ train/validation) within a single data
collection. But once they are applied to other pathologies, it becomes apparent
how much they overfit to the training pathologies (dice score drops between
$0.072\pm{0.001}$ for the left and $0.165\pm{0.001}$ for the right ventricle).
