  For the deployment of artificial intelligence (AI) in high-risk settings,
such as healthcare, methods that provide interpretability/explainability or
allow fine-grained error analysis are critical. Many recent methods for
interpretability/explainability and fine-grained error analysis use concepts,
which are meta-labels that are semantically meaningful to humans. However,
there are only a few datasets that include concept-level meta-labels and most
of these meta-labels are relevant for natural images that do not require domain
expertise. Densely annotated datasets in medicine focused on meta-labels that
are relevant to a single disease such as melanoma. In dermatology, skin disease
is described using an established clinical lexicon that allows clinicians to
describe physical exam findings to one another. To provide a medical dataset
densely annotated by domain experts with annotations useful across multiple
disease processes, we developed SkinCon: a skin disease dataset densely
annotated by dermatologists. SkinCon includes 3230 images from the Fitzpatrick
17k dataset densely annotated with 48 clinical concepts, 22 of which have at
least 50 images representing the concept. The concepts used were chosen by two
dermatologists considering the clinical descriptor terms used to describe skin
lesions. Examples include "plaque", "scale", and "erosion". The same concepts
were also used to label 656 skin disease images from the Diverse Dermatology
Images dataset, providing an additional external dataset with diverse skin tone
representations. We review the potential applications for the SkinCon dataset,
such as probing models, concept-based explanations, and concept bottlenecks.
Furthermore, we use SkinCon to demonstrate two of these use cases: debugging
mistakes of an existing dermatology AI model with concepts and developing
interpretable models with post-hoc concept bottleneck models.
