  Deep reinforcement learning has led to numerous notable results in robotics.
However, deep neural networks (DNNs) are unintuitive, which makes it difficult
to understand their predictions and strongly limits their potential for
real-world applications due to economic, safety, and assurance reasons. To
remedy this problem, a number of explainable AI methods have been presented,
such as SHAP and LIME, but these can be either be too costly to be used in
real-time robotic applications or provide only local explanations. In this
paper, the main contribution is the use of a linear model tree (LMT) to
approximate a DNN policy, originally trained via proximal policy
optimization(PPO), for an autonomous surface vehicle with five control inputs
performing a docking operation. The two main benefits of the proposed approach
are: a) LMTs are transparent which makes it possible to associate directly the
outputs (control actions, in our case) with specific values of the input
features, b) LMTs are computationally efficient and can provide information in
real-time. In our simulations, the opaque DNN policy controls the vehicle and
the LMT runs in parallel to provide explanations in the form of feature
attributions. Our results indicate that LMTs can be a useful component within
digital assurance frameworks for autonomous ships.
