  The key challenge of generative Visual Dialogue (VD) systems is to respond to
human queries with informative answers in natural and contiguous conversation
flow. Traditional Maximum Likelihood Estimation (MLE)-based methods only learn
from positive responses but ignore the negative responses, and consequently
tend to yield safe or generic responses. To address this issue, we propose a
novel training scheme in conjunction with weighted likelihood estimation (WLE)
method. Furthermore, an adaptive multi-modal reasoning module is designed, to
accommodate various dialogue scenarios automatically and select relevant
information accordingly. The experimental results on the VisDial benchmark
demonstrate the superiority of our proposed algorithm over other
state-of-the-art approaches, with an improvement of 5.81% on recall@10.
