  Recent studies have witnessed the effectiveness of 3D convolutions on
segmenting volumetric medical images. Compared with the 2D counterparts, 3D
convolutions can capture the spatial context in three dimensions. Nevertheless,
models employing 3D convolutions introduce more trainable parameters and are
more computationally complex, which may lead easily to model overfitting
especially for medical applications with limited available training data. This
paper aims to improve the effectiveness and efficiency of 3D convolutions by
introducing a novel Group Shift Pointwise Convolution (GSP-Conv). GSP-Conv
simplifies 3D convolutions into pointwise ones with 1x1x1 kernels, which
dramatically reduces the number of model parameters and FLOPs (e.g. 27x fewer
than 3D convolutions with 3x3x3 kernels). Na\"ive pointwise convolutions with
limited receptive fields cannot make full use of the spatial image context. To
address this problem, we propose a parameter-free operation, Group Shift (GS),
which shifts the feature maps along with different spatial directions in an
elegant way. With GS, pointwise convolutions can access features from different
spatial locations, and the limited receptive fields of pointwise convolutions
can be compensated. We evaluate the proposed methods on two datasets, PROMISE12
and BraTS18. Results show that our method, with substantially decreased model
complexity, achieves comparable or even better performance than models
employing 3D convolutions.
