  Interpretable machine learning has emerged as central in leveraging
artificial intelligence within high-stakes domains such as healthcare, where
understanding the rationale behind model predictions is as critical as
achieving high predictive accuracy. In this context, feature selection assumes
a pivotal role in enhancing model interpretability by identifying the most
important input features in black-box models. While random forests are
frequently used in biomedicine for their remarkable performance on tabular
datasets, the accuracy gained from aggregating decision trees comes at the
expense of interpretability. Consequently, feature selection for enhancing
interpretability in random forests has been extensively explored in supervised
settings. However, its investigation in the unsupervised regime remains notably
limited. To address this gap, the study introduces novel methods to construct
feature graphs from unsupervised random forests and feature selection
strategies to derive effective feature combinations from these graphs. Feature
graphs are constructed for the entire dataset as well as individual clusters
leveraging the parent-child node splits within the trees, such that feature
centrality captures their relevance to the clustering task, while edge weights
reflect the discriminating power of feature pairs. Graph-based feature
selection methods are extensively evaluated on synthetic and benchmark datasets
both in terms of their ability to reduce dimensionality while improving
clustering performance, as well as to enhance model interpretability. An
application on omics data for disease subtyping identifies the top features for
each cluster, showcasing the potential of the proposed approach to enhance
interpretability in clustering analyses and its utility in a real-world
biomedical application.
