  Assistance games (also known as cooperative inverse reinforcement learning
games) have been proposed as a model for beneficial AI, wherein a robotic agent
must act on behalf of a human principal but is initially uncertain about the
humans payoff function. This paper studies multi-principal assistance games,
which cover the more general case in which the robot acts on behalf of N humans
who may have widely differing payoffs. Impossibility theorems in social choice
theory and voting theory can be applied to such games, suggesting that
strategic behavior by the human principals may complicate the robots task in
learning their payoffs. We analyze in particular a bandit apprentice game in
which the humans act first to demonstrate their individual preferences for the
arms and then the robot acts to maximize the sum of human payoffs. We explore
the extent to which the cost of choosing suboptimal arms reduces the incentive
to mislead, a form of natural mechanism design. In this context we propose a
social choice method that uses shared control of a system to combine preference
inference with social welfare optimization.
