  The importance of explainability in AI has become a pressing concern, for
which several explainable AI (XAI) approaches have been recently proposed.
However, most of the available XAI techniques are post-hoc methods, which
however may be only partially reliable, as they do not reflect exactly the
state of the original models. Thus, a more direct way for achieving XAI is
through interpretable (also called glass-box) models. These models have been
shown to obtain comparable (and, in some cases, better) performance with
respect to black-boxes models in various tasks such as classification and
reinforcement learning. However, they struggle when working with raw data,
especially when the input dimensionality increases and the raw inputs alone do
not give valuable insights on the decision-making process. Here, we propose to
use end-to-end pipelines composed of multiple interpretable models co-optimized
by means of evolutionary algorithms, that allows us to decompose the
decision-making process into two parts: computing high-level features from raw
data, and reasoning on the extracted high-level features. We test our approach
in reinforcement learning environments from the Atari benchmark, where we
obtain comparable results (with respect to black-box approaches) in settings
without stochastic frame-skipping, while performance degrades in frame-skipping
settings.
