  This work summarizes the results of the largest skin image analysis challenge
in the world, hosted by the International Skin Imaging Collaboration (ISIC), a
global partnership that has organized the world's largest public repository of
dermoscopic images of skin. The challenge was hosted in 2018 at the Medical
Image Computing and Computer Assisted Intervention (MICCAI) conference in
Granada, Spain. The dataset included over 12,500 images across 3 tasks. 900
users registered for data download, 115 submitted to the lesion segmentation
task, 25 submitted to the lesion attribute detection task, and 159 submitted to
the disease classification task. Novel evaluation protocols were established,
including a new test for segmentation algorithm performance, and a test for
algorithm ability to generalize. Results show that top segmentation algorithms
still fail on over 10% of images on average, and algorithms with equal
performance on test data can have different abilities to generalize. This is an
important consideration for agencies regulating the growing set of machine
learning tools in the healthcare domain, and sets a new standard for future
public challenges in healthcare.
