  Even with the growing interest in problems at the intersection of Computer
Vision and Natural Language, grounding (i.e. identifying) the components of a
structured description in an image still remains a challenging task. This
contribution aims to propose a model which learns grounding by reconstructing
the visual features for the Multi-modal translation task. Previous works have
partially investigated standard approaches such as regression methods to
approximate the reconstruction of a visual input. In this paper, we propose a
different and novel approach which learns grounding by adversarial feedback. To
do so, we modulate our network following the recent promising adversarial
architectures and evaluate how the adversarial response from a visual
reconstruction as an auxiliary task helps the model in its learning. We report
the highest scores in term of BLEU and METEOR metrics on the different
datasets.
