  Long lasting efforts have been made to reduce radiation dose and thus the
potential radiation risk to the patient for computed tomography acquisitions
without severe deterioration of image quality. To this end, numerous
reconstruction and noise reduction algorithms have been developed, many of
which are based on iterative reconstruction techniques, incorporating prior
knowledge in the projection or image domain. Recently, deep learning-based
methods became increasingly popular and a multitude of papers claim ever
improving performance both quantitatively and qualitatively. In this work, we
find that the lack of a common benchmark setup and flaws in the experimental
setup of many publications hinder verifiability of those claims. We propose a
benchmark setup to overcome those flaws and improve reproducibility and
verifiability of experimental results in the field. In a comprehensive and fair
evaluation of several deep learning-based low dose CT denoising algorithms, we
find that most methods perform statistically similar and improvements over the
past six years have been marginal at best.
