  Deep learning has emerged as a powerful artificial intelligence tool to
interpret medical images for a growing variety of applications. However, the
paucity of medical imaging data with high-quality annotations that is necessary
for training such methods ultimately limits their performance. Medical data is
challenging to acquire due to privacy issues, shortage of experts available for
annotation, limited representation of rare conditions and cost. This problem
has previously been addressed by using synthetically generated data. However,
networks trained on synthetic data often fail to generalize to real data.
Cinematic rendering simulates the propagation and interaction of light passing
through tissue models reconstructed from CT data, enabling the generation of
photorealistic images. In this paper, we present one of the first applications
of cinematic rendering in deep learning, in which we propose to fine-tune
synthetic data-driven networks using cinematically rendered CT data for the
task of monocular depth estimation in endoscopy. Our experiments demonstrate
that: (a) Convolutional Neural Networks (CNNs) trained on synthetic data and
fine-tuned on photorealistic cinematically rendered data adapt better to real
medical images and demonstrate more robust performance when compared to
networks with no fine-tuning, (b) these fine-tuned networks require less
training data to converge to an optimal solution, and (c) fine-tuning with data
from a variety of photorealistic rendering conditions of the same scene
prevents the network from learning patient-specific information and aids in
generalizability of the model. Our empirical evaluation demonstrates that
networks fine-tuned with cinematically rendered data predict depth with 56.87%
less error for rendered endoscopy images and 27.49% less error for real porcine
colon endoscopy images.
