  Interactive Object Grasping (IOG) is the task of identifying and grasping the
desired object via human-robot natural language interaction. Current IOG
systems assume that a human user initially specifies the target object's
category (e.g., bottle). Inspired by pragmatics, where humans often convey
their intentions by relying on context to achieve goals, we introduce a new IOG
task, Pragmatic-IOG, and the corresponding dataset, Intention-oriented
Multi-modal Dialogue (IM-Dial). In our proposed task scenario, an
intention-oriented utterance (e.g., "I am thirsty") is initially given to the
robot. The robot should then identify the target object by interacting with a
human user. Based on the task setup, we propose a new robotic system that can
interpret the user's intention and pick up the target object, Pragmatic Object
Grasping (PROGrasp). PROGrasp performs Pragmatic-IOG by incorporating modules
for visual grounding, question asking, object grasping, and most importantly,
answer interpretation for pragmatic inference. Experimental results show that
PROGrasp is effective in offline (i.e., target object discovery) and online
(i.e., IOG with a physical robot arm) settings. Code and data are available at
https://github.com/gicheonkang/prograsp.
