  Recently, there has been increasing interest in transparency and
interpretability in Deep Reinforcement Learning (DRL) systems. Verbal
explanations, as the most natural way of communication in our daily life,
deserve more attention, since they allow users to gain a better understanding
of the system which ultimately could lead to a high level of trust and smooth
collaboration. This paper reports a novel work in generating verbal
explanations for DRL behaviors agent. A rule-based model is designed to
construct explanations using a series of rules which are predefined with prior
knowledge. A learning model is then proposed to expand the implicit logic of
generating verbal explanation to general situations by employing rule-based
explanations as training data. The learning model is shown to have better
flexibility and generalizability than the static rule-based model. The
performance of both models is evaluated quantitatively through objective
metrics. The results show that verbal explanation generated by both models
improve subjective satisfaction of users towards the interpretability of DRL
systems. Additionally, seven variants of the learning model are designed to
illustrate the contribution of input channels, attention mechanism, and
proposed encoder in improving the quality of verbal explanation.
