  Objective: To assess automatic computer-aided in-situ recognition of
morphological features of pure and mixed urinary stones using intraoperative
digital endoscopic images acquired in a clinical setting. Materials and
methods: In this single-centre study, an experienced urologist intraoperatively
and prospectively examined the surface and section of all kidney stones
encountered. Calcium oxalate monohydrate (COM/Ia), dihydrate (COD/IIb) and uric
acid (UA/IIIb) morphological criteria were collected and classified to generate
annotated datasets. A deep convolutional neural network (CNN) was trained to
predict the composition of both pure and mixed stones. To explain the
predictions of the deep neural network model, coarse localisation heat-maps
were plotted to pinpoint key areas identified by the network. Results: This
study included 347 and 236 observations of stone surface and stone section,
respectively. A highest sensitivity of 98 % was obtained for the type "pure
IIIb/UA" using surface images. The most frequently encountered morphology was
that of the type "pure Ia/COM"; it was correctly predicted in 91 % and 94 % of
cases using surface and section images, respectively. Of the mixed type
"Ia/COM+IIb/COD", Ia/COM was predicted in 84 % of cases using surface images,
IIb/COD in 70 % of cases, and both in 65 % of cases. Concerning mixed
Ia/COM+IIIb/UA stones, Ia/COM was predicted in 91 % of cases using section
images, IIIb/UA in 69 % of cases, and both in 74 % of cases. Conclusions: This
preliminary study demonstrates that deep convolutional neural networks are
promising to identify kidney stone composition from endoscopic images acquired
intraoperatively. Both pure and mixed stone composition could be discriminated.
Collected in a clinical setting, surface and section images analysed by deep
CNN provide valuable information about stone morphology for computer-aided
diagnosis.
