  Interactive visual grounding in Human-Robot Interaction (HRI) is challenging
yet practical due to the inevitable ambiguity in natural languages. It requires
robots to disambiguate the user input by active information gathering. Previous
approaches often rely on predefined templates to ask disambiguation questions,
resulting in performance reduction in realistic interactive scenarios. In this
paper, we propose TiO, an end-to-end system for interactive visual grounding in
human-robot interaction. Benefiting from a unified formulation of visual
dialogue and grounding, our method can be trained on a joint of extensive
public data, and show superior generality to diversified and challenging
open-world scenarios. In the experiments, we validate TiO on GuessWhat?! and
InViG benchmarks, setting new state-of-the-art performance by a clear margin.
Moreover, we conduct HRI experiments on the carefully selected 150 challenging
scenes as well as real-robot platforms. Results show that our method
demonstrates superior generality to diversified visual and language inputs with
a high success rate. Codes and demos are available at
https://github.com/jxu124/TiO.
