  This paper presents a systematic literature review (SLR) on the
explainability and interpretability of machine learning (ML) models within the
context of predictive process mining, using the PRISMA framework. Given the
rapid advancement of artificial intelligence (AI) and ML systems, understanding
the "black-box" nature of these technologies has become increasingly critical.
Focusing specifically on the domain of process mining, this paper delves into
the challenges of interpreting ML models trained with complex business process
data. We differentiate between intrinsically interpretable models and those
that require post-hoc explanation techniques, providing a comprehensive
overview of the current methodologies and their applications across various
application domains. Through a rigorous bibliographic analysis, this research
offers a detailed synthesis of the state of explainability and interpretability
in predictive process mining, identifying key trends, challenges, and future
directions. Our findings aim to equip researchers and practitioners with a
deeper understanding of how to develop and implement more trustworthy,
transparent, and effective intelligent systems for predictive process
analytics.
