  The clinical diagnosis of skin lesion involves the analysis of dermoscopic
and clinical modalities. Dermoscopic images provide a detailed view of the
surface structures whereas clinical images offer a complementary macroscopic
information. The visual diagnosis of melanoma is also based on seven-point
checklist which involves identifying different visual attributes. Recently,
supervised learning approaches such as convolutional neural networks (CNNs)
have shown great performances using both dermoscopic and clinical modalities
(Multi-modality). The seven different visual attributes in the checklist are
also used to further improve the the diagnosis. The performances of these
approaches, however, are still reliant on the availability of large-scaled
labeled data. The acquisition of annotated dataset is an expensive and
time-consuming task, more so with annotating multi-attributes. To overcome this
limitation, we propose a self-supervised learning (SSL) algorithm for
multi-modality skin lesion classification. Our algorithm enables the
multi-modality learning by maximizing the similarities between paired
dermoscopic and clinical images from different views. In addition, we generate
surrogate pseudo-multi-labels that represent seven attributes via clustering
analysis. We also propose a label-relation-aware module to refine each
pseudo-label embedding and capture the interrelationships between
pseudo-multi-labels. We validated the effectiveness of our algorithm using
well-benchmarked seven-point skin lesion dataset. Our results show that our
algorithm achieved better performances than other state-of-the-art SSL
counterparts.
