  Melanoma is regarded as the most threatening among all skin cancers. There is
a pressing need to build systems which can aid in the early detection of
melanoma and enable timely treatment to patients. Recent methods are geared
towards machine learning based systems where the task is posed as image
recognition, tag dermoscopic images of skin lesions as melanoma or
non-melanoma. Even though these methods show promising results in terms of
accuracy, they are computationally quite expensive to train, that questions the
ability of these models to be deployable in a clinical setting or memory
constraint devices. To address this issue, we focus on building simple and
performant models having few layers, less than ten compared to hundreds. As
well as with fewer learnable parameters, 0.26 million (M) compared to 42.5M
using knowledge distillation with the goal to detect melanoma from dermoscopic
images. First, we train a teacher model using a ResNet-50 to detect melanoma.
Using the teacher model, we train the student model known as Distilled Student
Network (DSNet) which has around 0.26M parameters using knowledge distillation
achieving an accuracy of 91.7%. We compare against ImageNet pre-trained models
such MobileNet, VGG-16, Inception-V3, EfficientNet-B0, ResNet-50 and
ResNet-101. We find that our approach works well in terms of inference runtime
compared to other pre-trained models, 2.57 seconds compared to 14.55 seconds.
We find that DSNet (0.26M parameters), which is 15 times smaller, consistently
performs better than EfficientNet-B0 (4M parameters) in both melanoma and
non-melanoma detection across Precision, Recall and F1 scores
