  Despite tremendous progress, machine learning and deep learning still suffer
from incomprehensible predictions. Incomprehensibility, however, is not an
option for the use of (deep) reinforcement learning in the real world, as
unpredictable actions can seriously harm the involved individuals. In this
work, we propose a genetic programming framework to generate explanations for
the decision-making process of already trained agents by imitating them with
programs. Programs are interpretable and can be executed to generate
explanations of why the agent chooses a particular action. Furthermore, we
conduct an ablation study that investigates how extending the domain-specific
language by using library learning alters the performance of the method. We
compare our results with the previous state of the art for this problem and
show that we are comparable in performance but require much less hardware
resources and computation time.
