  Recently deep residual learning with residual units for training very deep
neural networks advanced the state-of-the-art performance on 2D image
recognition tasks, e.g., object detection and segmentation. However, how to
fully leverage contextual representations for recognition tasks from volumetric
data has not been well studied, especially in the field of medical image
computing, where a majority of image modalities are in volumetric format. In
this paper we explore the deep residual learning on the task of volumetric
brain segmentation. There are at least two main contributions in our work.
First, we propose a deep voxelwise residual network, referred as VoxResNet,
which borrows the spirit of deep residual learning in 2D image recognition
tasks, and is extended into a 3D variant for handling volumetric data. Second,
an auto-context version of VoxResNet is proposed by seamlessly integrating the
low-level image appearance features, implicit shape information and high-level
context together for further improving the volumetric segmentation performance.
Extensive experiments on the challenging benchmark of brain segmentation from
magnetic resonance (MR) images corroborated the efficacy of our proposed method
in dealing with volumetric data. We believe this work unravels the potential of
3D deep learning to advance the recognition performance on volumetric image
segmentation.
