  Deep convolutional neural networks (CNNs), especially fully convolutional
networks, have been widely applied to automatic medical image segmentation
problems, e.g., multi-organ segmentation. Existing CNN-based segmentation
methods mainly focus on looking for increasingly powerful network
architectures, but pay less attention to data sampling strategies for training
networks more effectively. In this paper, we present a simple but effective
sample selection method for training multi-organ segmentation networks. Sample
selection exhibits an exploitation-exploration strategy, i.e., exploiting hard
samples and exploring less frequently visited samples. Based on the fact that
very hard samples might have annotation errors, we propose a new sample
selection policy, named Relaxed Upper Confident Bound (RUCB). Compared with
other sample selection policies, e.g., Upper Confident Bound (UCB), it exploits
a range of hard samples rather than being stuck with a small set of very hard
ones, which mitigates the influence of annotation errors during training. We
apply this new sample selection policy to training a multi-organ segmentation
network on a dataset containing 120 abdominal CT scans and show that it boosts
segmentation performance significantly.
