  The task of localizing and categorizing objects in medical images often
remains formulated as a semantic segmentation problem. This approach, however,
only indirectly solves the coarse localization task by predicting pixel-level
scores, requiring ad-hoc heuristics when mapping back to object-level scores.
State-of-the-art object detectors on the other hand, allow for individual
object scoring in an end-to-end fashion, while ironically trading in the
ability to exploit the full pixel-wise supervision signal. This can be
particularly disadvantageous in the setting of medical image analysis, where
data sets are notoriously small. In this paper, we propose Retina U-Net, a
simple architecture, which naturally fuses the Retina Net one-stage detector
with the U-Net architecture widely used for semantic segmentation in medical
images. The proposed architecture recaptures discarded supervision signals by
complementing object detection with an auxiliary task in the form of semantic
segmentation without introducing the additional complexity of previously
proposed two-stage detectors. We evaluate the importance of full segmentation
supervision on two medical data sets, provide an in-depth analysis on a series
of toy experiments and show how the corresponding performance gain grows in the
limit of small data sets. Retina U-Net yields strong detection performance only
reached by its more complex two-staged counterparts. Our framework including
all methods implemented for operation on 2D and 3D images is available at
github.com/pfjaeger/medicaldetectiontoolkit.
