  Confocal laser endomicroscopy (CLE) is an advanced optical fluorescence
technology undergoing assessment for applications in brain tumor surgery.
Despite its promising potential, interpreting the unfamiliar gray tone images
of fluorescent stains can be difficult. Many of the CLE images can be distorted
by motion, extremely low or high fluorescence signal, or obscured by red blood
cell accumulation, and these can be interpreted as nondiagnostic. However, just
one neat CLE image might suffice for intraoperative diagnosis of the tumor.
While manual examination of thousands of nondiagnostic images during surgery
would be impractical, this creates an opportunity for a model to select
diagnostic images for the pathologists or surgeon's review. In this study, we
sought to develop a deep learning model to automatically detect the diagnostic
images using a manually annotated dataset, and we employed a patient-based
nested cross-validation approach to explore generalizability of the model. We
explored various training regimes: deep training, shallow fine-tuning, and deep
fine-tuning. Further, we investigated the effect of ensemble modeling by
combining the top-5 single models crafted in the development phase. We
localized histological features from diagnostic CLE images by visualization of
shallow and deep neural activations. Our inter-rater experiment results
confirmed that our ensemble of deeply fine-tuned models achieved higher
agreement with the ground truth than the other observers. With the speed and
precision of the proposed method (110 images/second; 85% on the gold standard
test subset), it has potential to be integrated into the operative workflow in
the brain tumor surgery.
