  Machine learning-based approaches outperform competing methods in most
disciplines relevant to diagnostic radiology. Interventional radiology,
however, has not yet benefited substantially from the advent of deep learning,
in particular because of two reasons: 1) Most images acquired during the
procedure are never archived and are thus not available for learning, and 2)
even if they were available, annotations would be a severe challenge due to the
vast amounts of data. When considering fluoroscopy-guided procedures, an
interesting alternative to true interventional fluoroscopy is in silico
simulation of the procedure from 3D diagnostic CT. In this case, labeling is
comparably easy and potentially readily available, yet, the appropriateness of
resulting synthetic data is dependent on the forward model. In this work, we
propose DeepDRR, a framework for fast and realistic simulation of fluoroscopy
and digital radiography from CT scans, tightly integrated with the software
platforms native to deep learning. We use machine learning for material
decomposition and scatter estimation in 3D and 2D, respectively, combined with
analytic forward projection and noise injection to achieve the required
performance. On the example of anatomical landmark detection in X-ray images of
the pelvis, we demonstrate that machine learning models trained on DeepDRRs
generalize to unseen clinically acquired data without the need for re-training
or domain adaptation. Our results are promising and promote the establishment
of machine learning in fluoroscopy-guided procedures.
