  Supervised machine learning explainability has developed rapidly in recent
years. However, clustering explainability has lagged behind. Here, we
demonstrate the first adaptation of model-agnostic explainability methods to
explain unsupervised clustering. We present two novel "algorithm-agnostic"
explainability methods - global permutation percent change (G2PC) and local
perturbation percent change (L2PC) - that identify feature importance globally
to a clustering algorithm and locally to the clustering of individual samples.
The methods are (1) easy to implement and (2) broadly applicable across
clustering algorithms, which could make them highly impactful. We demonstrate
the utility of the methods for explaining five popular clustering methods on
low-dimensional synthetic datasets and on high-dimensional functional network
connectivity data extracted from a resting-state functional magnetic resonance
imaging dataset of 151 individuals with schizophrenia and 160 controls. Our
results are consistent with existing literature while also shedding new light
on how changes in brain connectivity may lead to schizophrenia symptoms. We
further compare the explanations from our methods to an interpretable
classifier and find them to be highly similar. Our proposed methods robustly
explain multiple clustering algorithms and could facilitate new insights into
many applications. We hope this study will greatly accelerate the development
of the field of clustering explainability.
