  This paper examines the effect of real-time, personalized alignment of a
robot's reward function to the human's values on trust and team performance. We
present and compare three distinct robot interaction strategies: a non-learner
strategy where the robot presumes the human's reward function mirrors its own,
a non-adaptive-learner strategy in which the robot learns the human's reward
function for trust estimation and human behavior modeling, but still optimizes
its own reward function, and an adaptive-learner strategy in which the robot
learns the human's reward function and adopts it as its own. Two human-subject
experiments with a total number of 54 participants were conducted. In both
experiments, the human-robot team searches for potential threats in a town. The
team sequentially goes through search sites to look for threats. We model the
interaction between the human and the robot as a trust-aware Markov Decision
Process (trust-aware MDP) and use Bayesian Inverse Reinforcement Learning (IRL)
to estimate the reward weights of the human as they interact with the robot. In
Experiment 1, we start our learning algorithm with an informed prior of the
human's values/goals. In Experiment 2, we start the learning algorithm with an
uninformed prior. Results indicate that when starting with a good informed
prior, personalized value alignment does not seem to benefit trust or team
performance. On the other hand, when an informed prior is unavailable,
alignment to the human's values leads to high trust and higher perceived
performance while maintaining the same objective team performance.
