  Adoption and deployment of robotic and autonomous systems in industry are
currently hindered by the lack of transparency, required for safety and
accountability. Methods for providing explanations are needed that are agnostic
to the underlying autonomous system and easily updated. Furthermore, different
stakeholders with varying levels of expertise, will require different levels of
information. In this work, we use surrogate models to provide transparency as
to the underlying policies for behaviour activation. We show that these
surrogate models can effectively break down autonomous agents' behaviour into
explainable components for use in natural language explanations.
