  This paper tackles the problem of learning a questioner in the goal-oriented
visual dialog task. Several previous works adopt model-free reinforcement
learning. Most pretrain the model from a finite set of human-generated data. We
argue that using limited demonstrations to kick-start the questioner is
insufficient due to the large policy search space. Inspired by a recently
proposed information theoretic approach, we develop two analytic experts to
serve as a source of high-quality demonstrations for imitation learning. We
then take advantage of reinforcement learning to refine the model towards the
goal-oriented objective. Experimental results on the GuessWhat?! dataset show
that our method has the combined merits of imitation and reinforcement
learning, achieving the state-of-the-art performance.
