  Tasks where robots must anticipate human intent, such as navigating around a
cluttered home or sorting everyday items, are challenging because they exhibit
a wide range of valid actions that lead to similar outcomes. Moreover,
zero-shot cooperation between human-robot partners is an especially challenging
problem because it requires the robot to infer and adapt on the fly to a latent
human intent, which could vary significantly from human to human. Recently,
deep learned motion prediction models have shown promising results in
predicting human intent but are prone to being confidently incorrect. In this
work, we present Risk-Calibrated Interactive Planning (RCIP), which is a
framework for measuring and calibrating risk associated with uncertain action
selection in human-robot cooperation, with the fundamental idea that the robot
should ask for human clarification when the risk associated with the
uncertainty in the human's intent cannot be controlled. RCIP builds on the
theory of set-valued risk calibration to provide a finite-sample statistical
guarantee on the cumulative loss incurred by the robot while minimizing the
cost of human clarification in complex multi-step settings. Our main insight is
to frame the risk control problem as a sequence-level multi-hypothesis testing
problem, allowing efficient calibration using a low-dimensional parameter that
controls a pre-trained risk-aware policy. Experiments across a variety of
simulated and real-world environments demonstrate RCIP's ability to predict and
adapt to a diverse set of dynamic human intents.
