  Explainability of Reinforcement Learning (RL) policies remains a challenging
research problem, particularly when considering RL in a safety context.
Understanding the decisions and intentions of an RL policy offer avenues to
incorporate safety into the policy by limiting undesirable actions. We propose
the use of a Boolean Decision Rules model to create a post-hoc rule-based
summary of an agent's policy. We evaluate our proposed approach using a DQN
agent trained on an implementation of a lava gridworld and show that, given a
hand-crafted feature representation of this gridworld, simple generalised rules
can be created, giving a post-hoc explainable summary of the agent's policy. We
discuss possible avenues to introduce safety into a RL agent's policy by using
rules generated by this rule-based model as constraints imposed on the agent's
policy, as well as discuss how creating simple rule summaries of an agent's
policy may help in the debugging process of RL agents.
