  The adoption of Reinforcement Learning (RL) in several human-centred
applications provides robots with autonomous decision-making capabilities and
adaptability based on the observations of the operating environment. In such
scenarios, however, the learning process can make robots' behaviours unclear
and unpredictable to humans, thus preventing a smooth and effective Human-Robot
Interaction (HRI). As a consequence, it becomes crucial to avoid robots
performing actions that are unclear to the user. In this work, we investigate
whether including human preferences in RL (concerning the actions the robot
performs during learning) improves the transparency of a robot's behaviours.
For this purpose, a shielding mechanism is included in the RL algorithm to
include human preferences and to monitor the learning agent's decisions. We
carried out a within-subjects study involving 26 participants to evaluate the
robot's transparency in terms of Legibility, Predictability, and Expectability
in different settings. Results indicate that considering human preferences
during learning improves Legibility with respect to providing only
Explanations, and combining human preferences with explanations elucidating the
rationale behind the robot's decisions further amplifies transparency. Results
also confirm that an increase in transparency leads to an increase in the
safety, comfort, and reliability of the robot. These findings show the
importance of transparency during learning and suggest a paradigm for robotic
applications with human in the loop.
