  Human-robot collaboration (HRC) has become increasingly relevant in
industrial, household, and commercial settings. However, the effectiveness of
such collaborations is highly dependent on the human and robots' situational
awareness of the environment. Improving this awareness includes not only
aligning perceptions in a shared workspace, but also bidirectionally
communicating intent and visualizing different states of the environment to
enhance scene understanding. In this paper, we propose ARDIE (Augmented Reality
with Dialogue and Eye Gaze), a novel intelligent agent that leverages
multi-modal feedback cues to enhance HRC. Our system utilizes a decision
theoretic framework to formulate a joint policy that incorporates interactive
augmented reality (AR), natural language, and eye gaze to portray current and
future states of the environment. Through object-specific AR renders, the human
can visualize future object interactions to make adjustments as needed,
ultimately providing an interactive and efficient collaboration between humans
and robots.
