  Dermatologists often diagnose or rule out early melanoma by evaluating the
follow-up dermoscopic images of skin lesions. However, existing algorithms for
early melanoma diagnosis are developed using single time-point images of
lesions. Ignoring the temporal, morphological changes of lesions can lead to
misdiagnosis in borderline cases. In this study, we propose a framework for
automated early melanoma diagnosis using sequential dermoscopic images. To this
end, we construct our method in three steps. First, we align sequential
dermoscopic images of skin lesions using estimated Euclidean transformations,
extract the lesion growth region by computing image differences among the
consecutive images, and then propose a spatio-temporal network to capture the
dermoscopic changes from aligned lesion images and the corresponding difference
images. Finally, we develop an early diagnosis module to compute probability
scores of malignancy for lesion images over time. We collected 179 serial
dermoscopic imaging data from 122 patients to verify our method. Extensive
experiments show that the proposed model outperforms other commonly used
sequence models. We also compared the diagnostic results of our model with
those of seven experienced dermatologists and five registrars. Our model
achieved higher diagnostic accuracy than clinicians (63.69% vs. 54.33%,
respectively) and provided an earlier diagnosis of melanoma (60.7% vs. 32.7% of
melanoma correctly diagnosed on the first follow-up images). These results
demonstrate that our model can be used to identify melanocytic lesions that are
at high-risk of malignant transformation earlier in the disease process and
thereby redefine what is possible in the early detection of melanoma.
