  Segmentation of white matter lesions and deep grey matter structures is an
important task in the quantification of magnetic resonance imaging in multiple
sclerosis. In this paper we explore segmentation solutions based on
convolutional neural networks (CNNs) for providing fast, reliable segmentations
of lesions and grey-matter structures in multi-modal MR imaging, and the
performance of these methods when applied to out-of-centre data.
  We trained two state-of-the-art fully convolutional CNN architectures on the
2016 MSSEG training dataset, which was annotated by seven independent human
raters: a reference implementation of a 3D Unet, and a more recently proposed
3D-to-2D architecture (DeepSCAN). We then retrained those methods on a larger
dataset from a single centre, with and without labels for other brain
structures. We quantified changes in performance owing to dataset shift, and
changes in performance by adding the additional brain-structure labels. We also
compared performance with freely available reference methods.
  Both fully-convolutional CNN methods substantially outperform other
approaches in the literature when trained and evaluated in cross-validation on
the MSSEG dataset, showing agreement with human raters in the range of human
inter-rater variability. Both architectures showed drops in performance when
trained on single-centre data and tested on the MSSEG dataset. When trained
with the addition of weak anatomical labels derived from Freesurfer, the
performance of the 3D Unet degraded, while the performance of the DeepSCAN net
improved. Overall, the DeepSCAN network predicting both lesion and anatomical
labels was the best-performing network examined.
