  Autonomous systems can substantially enhance a human's efficiency and
effectiveness in complex environments. Machines, however, are often unable to
observe the preferences of the humans that they serve. Despite the fact that
the human's and machine's objectives are aligned, asymmetric information, along
with heterogeneous sensitivities to risk by the human and machine, make their
joint optimization process a game with strategic interactions. We propose a
framework based on risk-sensitive dynamic games; the human seeks to optimize
her risk-sensitive criterion according to her true preferences, while the
machine seeks to adaptively learn the human's preferences and at the same time
provide a good service to the human. We develop a class of performance measures
for the proposed framework based on the concept of regret. We then evaluate
their dependence on the risk-sensitivity and the degree of uncertainty. We
present applications of our framework to self-driving taxis, and robo-financial
advising.
