  Appendicitis is among the most frequent reasons for pediatric abdominal
surgeries. Previous decision support systems for appendicitis have focused on
clinical, laboratory, scoring, and computed tomography data and have ignored
abdominal ultrasound, despite its noninvasive nature and widespread
availability. In this work, we present interpretable machine learning models
for predicting the diagnosis, management and severity of suspected appendicitis
using ultrasound images. Our approach utilizes concept bottleneck models (CBM)
that facilitate interpretation and interaction with high-level concepts
understandable to clinicians. Furthermore, we extend CBMs to prediction
problems with multiple views and incomplete concept sets. Our models were
trained on a dataset comprising 579 pediatric patients with 1709 ultrasound
images accompanied by clinical and laboratory data. Results show that our
proposed method enables clinicians to utilize a human-understandable and
intervenable predictive model without compromising performance or requiring
time-consuming image annotation when deployed. For predicting the diagnosis,
the extended multiview CBM attained an AUROC of 0.80 and an AUPR of 0.92,
performing comparably to similar black-box neural networks trained and tested
on the same dataset.
