  Humans are able to identify a referred visual object in a complex scene via a
few rounds of natural language communications. Success communication requires
both parties to engage and learn to adapt for each other. In this paper, we
introduce an interactive training method to improve the natural language
conversation system for a visual grounding task. During interactive training,
both agents are reinforced by the guidance from a common reward function. The
parametrized reward function also cooperatively updates itself via
interactions, and contribute to accomplishing the task. We evaluate the method
on GuessWhat?! visual grounding task, and significantly improve the task
success rate. However, we observe language drifting problem during training and
propose to use reward engineering to improve the interpretability for the
generated conversations. Our result also indicates evaluating goal-ended visual
conversation tasks require semantic relevant metrics beyond task success rate.
