  The computer-assisted radiologic informative report has received increasing
research attention to facilitate diagnosis and treatment planning for dental
care providers. However, manual interpretation of dental images is limited,
expensive, and time-consuming. Another barrier in dental imaging is the limited
number of available images for training, which is a challenge in the era of
deep learning. This study proposes a novel self-distillation (SD) enhanced
self-supervised learning on top of the masked image modeling (SimMIM)
Transformer, called SD-SimMIM, to improve the outcome with a limited number of
dental radiographs. In addition to the prediction loss on masked patches,
SD-SimMIM computes the self-distillation loss on the visible patches. We apply
SD-SimMIM on dental panoramic X-rays for teeth numbering, detection of dental
restorations and orthodontic appliances, and instance segmentation tasks. Our
results show that SD-SimMIM outperforms other self-supervised learning methods.
Furthermore, we augment and improve the annotation of an existing dataset of
panoramic X-rays.
