  Automated and accurate 3D medical image segmentation plays an essential role
in assisting medical professionals to evaluate disease progresses and make fast
therapeutic schedules. Although deep convolutional neural networks (DCNNs) have
widely applied to this task, the accuracy of these models still need to be
further improved mainly due to their limited ability to 3D context perception.
In this paper, we propose the 3D context residual network (ConResNet) for the
accurate segmentation of 3D medical images. This model consists of an encoder,
a segmentation decoder, and a context residual decoder. We design the context
residual module and use it to bridge both decoders at each scale. Each context
residual module contains both context residual mapping and context attention
mapping, the formal aims to explicitly learn the inter-slice context
information and the latter uses such context as a kind of attention to boost
the segmentation accuracy. We evaluated this model on the MICCAI 2018 Brain
Tumor Segmentation (BraTS) dataset and NIH Pancreas Segmentation (Pancreas-CT)
dataset. Our results not only demonstrate the effectiveness of the proposed 3D
context residual learning scheme but also indicate that the proposed ConResNet
is more accurate than six top-ranking methods in brain tumor segmentation and
seven top-ranking methods in pancreas segmentation. Code is available at
https://git.io/ConResNet
