  The Guesser is a task of visual grounding in GuessWhat?! like visual
dialogue. It locates the target object in an image supposed by an Oracle
oneself over a question-answer based dialogue between a Questioner and the
Oracle. Most existing guessers make one and only one guess after receiving all
question-answer pairs in a dialogue with the predefined number of rounds. This
paper proposes a guessing state for the Guesser, and regards guess as a process
with change of guessing state through a dialogue. A guessing state tracking
based guess model is therefore proposed. The guessing state is defined as a
distribution on objects in the image. With that in hand, two loss functions are
defined as supervisions for model training. Early supervision brings
supervision to Guesser at early rounds, and incremental supervision brings
monotonicity to the guessing state. Experimental results on GuessWhat?! dataset
show that our model significantly outperforms previous models, achieves new
state-of-the-art, especially the success rate of guessing 83.3% is approaching
the human-level accuracy of 84.4%.
