  Convolutional neural networks (CNNs) allow for parameter sharing and
translational equivariance by using convolutional kernels in their linear
layers. By restricting these kernels to be SO(3)-steerable, CNNs can further
improve parameter sharing. These rotationally-equivariant convolutional layers
have several advantages over standard convolutional layers, including increased
robustness to unseen poses, smaller network size, and improved sample
efficiency. Despite this, most segmentation networks used in medical image
analysis continue to rely on standard convolutional kernels. In this paper, we
present a new family of segmentation networks that use equivariant voxel
convolutions based on spherical harmonics. These networks are robust to data
poses not seen during training, and do not require rotation-based data
augmentation during training. In addition, we demonstrate improved segmentation
performance in MRI brain tumor and healthy brain structure segmentation tasks,
with enhanced robustness to reduced amounts of training data and improved
parameter efficiency. Code to reproduce our results, and to implement the
equivariant segmentation networks for other tasks is available at
http://github.com/SCAN-NRAD/e3nn_Unet
