  Current one-stage methods for visual grounding encode the language query as
one holistic sentence embedding before fusion with visual feature. Such a
formulation does not treat each word of a query sentence on par when modeling
language to visual attention, therefore prone to neglect words which are less
important for sentence embedding but critical for visual grounding. In this
paper we propose Word2Pix: a one-stage visual grounding network based on
encoder-decoder transformer architecture that enables learning for textual to
visual feature correspondence via word to pixel attention. The embedding of
each word from the query sentence is treated alike by attending to visual
pixels individually instead of single holistic sentence embedding. In this way,
each word is given equivalent opportunity to adjust the language to vision
attention towards the referent target through multiple stacks of transformer
decoder layers. We conduct the experiments on RefCOCO, RefCOCO+ and RefCOCOg
datasets and the proposed Word2Pix outperforms existing one-stage methods by a
notable margin. The results obtained also show that Word2Pix surpasses
two-stage visual grounding models, while at the same time keeping the merits of
one-stage paradigm namely end-to-end training and real-time inference speed
intact.
