  A neural multimodal machine translation (MMT) system is one that aims to
perform better translation by extending conventional text-only translation
models with multimodal information. Many recent studies report improvements
when equipping their models with the multimodal module, despite the controversy
of whether such improvements indeed come from the multimodal part. We revisit
the contribution of multimodal information in MMT by devising two interpretable
MMT models. To our surprise, although our models replicate similar gains as
recently developed multimodal-integrated systems achieved, our models learn to
ignore the multimodal information. Upon further investigation, we discover that
the improvements achieved by the multimodal models over text-only counterparts
are in fact results of the regularization effect. We report empirical findings
that highlight the importance of MMT models' interpretability, and discuss how
our findings will benefit future research.
