  A method for obtaining appropriate reaction coordinates is required to
identify transition states distinguishing product and reactant in complex
molecular systems. Recently, abundant research has been devoted to obtaining
reaction coordinates using artificial neural networks from deep learning
literature, where many collective variables are typically utilized in the input
layer. However, it is difficult to explain the details of which collective
variables contribute to the predicted reaction coordinates owing to the
complexity of the nonlinear functions in deep neural networks. To overcome this
limitation, we used Explainable Artificial Intelligence (XAI) methods of the
Local Interpretable Model-agnostic Explanation (LIME) and the game theory-based
framework known as Shapley Additive exPlanations (SHAP). We demonstrated that
XAI enables us to obtain the degree of contribution of each collective variable
to reaction coordinates that is determined by nonlinear regressions with deep
learning for the committor of the alanine dipeptide isomerization in vacuum. In
particular, both LIME and SHAP provide important features to the predicted
reaction coordinates, which are characterized by appropriate dihedral angles
consistent with those previously reported from the committor test analysis. The
present study offers an AI-aided framework to explain the appropriate reaction
coordinates, which acquires considerable significance when the number of
degrees of freedom increases.
