  With the introduction of fully convolutional neural networks, deep learning
has raised the benchmark for medical image segmentation on both speed and
accuracy, and different networks have been proposed for 2D and 3D segmentation
with promising results. Nevertheless, most networks only handle relatively
small numbers of labels (<10), and there are very limited works on handling
highly unbalanced object sizes especially in 3D segmentation. In this paper, we
propose a network architecture and the corresponding loss function which
improve segmentation of very small structures. By combining skip connections
and deep supervision with respect to the computational feasibility of 3D
segmentation, we propose a fast converging and computationally efficient
network architecture for accurate segmentation. Furthermore, inspired by the
concept of focal loss, we propose an exponential logarithmic loss which
balances the labels not only by their relative sizes but also by their
segmentation difficulties. We achieve an average Dice coefficient of 82% on
brain segmentation with 20 labels, with the ratio of the smallest to largest
object sizes as 0.14%. Less than 100 epochs are required to reach such
accuracy, and segmenting a 128x128x128 volume only takes around 0.4 s.
