  Explainable Artificial Intelligence (XAI), i.e., the development of more
transparent and interpretable AI models, has gained increased traction over the
last few years. This is due to the fact that, in conjunction with their growth
into powerful and ubiquitous tools, AI models exhibit one detrimential
characteristic: a performance-transparency trade-off. This describes the fact
that the more complex a model's inner workings, the less clear it is how its
predictions or decisions were achieved. But, especially considering Machine
Learning (ML) methods like Reinforcement Learning (RL) where the system learns
autonomously, the necessity to understand the underlying reasoning for their
decisions becomes apparent. Since, to the best of our knowledge, there exists
no single work offering an overview of Explainable Reinforcement Learning (XRL)
methods, this survey attempts to address this gap. We give a short summary of
the problem, a definition of important terms, and offer a classification and
assessment of current XRL methods. We found that a) the majority of XRL methods
function by mimicking and simplifying a complex model instead of designing an
inherently simple one, and b) XRL (and XAI) methods often neglect to consider
the human side of the equation, not taking into account research from related
fields like psychology or philosophy. Thus, an interdisciplinary effort is
needed to adapt the generated explanations to a (non-expert) human user in
order to effectively progress in the field of XRL and XAI in general.
