  Interpretability is a crucial aspect of machine learning models that enables
humans to understand and trust the decision-making process of these models. In
many real-world applications, the interpretability of models is essential for
legal, ethical, and practical reasons. For instance, in the banking domain,
interpretability is critical for lenders and borrowers to understand the
reasoning behind the acceptance or rejection of loan applications as per fair
lending laws. However, achieving interpretability in machine learning models is
challenging, especially for complex high-performance models. Hence Explainable
Boosting Machines (EBMs) have been gaining popularity due to their
interpretable and high-performance nature in various prediction tasks. However,
these models can suffer from issues such as spurious interactions with
redundant features and single-feature dominance across all interactions, which
can affect the interpretability and reliability of the model's predictions. In
this paper, we explore novel approaches to address these issues by utilizing
alternate Cross-feature selection, ensemble features and model configuration
alteration techniques. Our approach involves a multi-step feature selection
procedure that selects a set of candidate features, ensemble features and then
benchmark the same using the EBM model. We evaluate our method on three
benchmark datasets and show that the alternate techniques outperform vanilla
EBM methods, while providing better interpretability and feature selection
stability, and improving the model's predictive performance. Moreover, we show
that our approach can identify meaningful interactions and reduce the dominance
of single features in the model's predictions, leading to more reliable and
interpretable models.
  Index Terms- Interpretability, EBM's, ensemble, feature selection.
