  Automatic detection of pulmonary nodules in thoracic computed tomography (CT)
scans has been an active area of research for the last two decades. However,
there have only been few studies that provide a comparative performance
evaluation of different systems on a common database. We have therefore set up
the LUNA16 challenge, an objective evaluation framework for automatic nodule
detection algorithms using the largest publicly available reference database of
chest CT scans, the LIDC-IDRI data set. In LUNA16, participants develop their
algorithm and upload their predictions on 888 CT scans in one of the two
tracks: 1) the complete nodule detection track where a complete CAD system
should be developed, or 2) the false positive reduction track where a provided
set of nodule candidates should be classified. This paper describes the setup
of LUNA16 and presents the results of the challenge so far. Moreover, the
impact of combining individual systems on the detection performance was also
investigated. It was observed that the leading solutions employed convolutional
networks and used the provided set of nodule candidates. The combination of
these solutions achieved an excellent sensitivity of over 95% at fewer than 1.0
false positives per scan. This highlights the potential of combining algorithms
to improve the detection performance. Our observer study with four expert
readers has shown that the best system detects nodules that were missed by
expert readers who originally annotated the LIDC-IDRI data. We released this
set of additional nodules for further development of CAD systems.
