  Having the difficulty of solving the semantic gap between images and texts
for the image captioning task, conventional studies in this area paid some
attention to treating semantic concepts as a bridge between the two modalities
and improved captioning performance accordingly. Although promising results on
concept prediction were obtained, the aforementioned studies normally ignore
the relationship among concepts, which relies on not only objects in the image,
but also word dependencies in the text, so that offers a considerable potential
for improving the process of generating good descriptions. In this paper, we
propose a structured concept predictor (SCP) to predict concepts and their
structures, then we integrate them into captioning, so as to enhance the
contribution of visual signals in this task via concepts and further use their
relations to distinguish cross-modal semantics for better description
generation. Particularly, we design weighted graph convolutional networks
(W-GCN) to depict concept relations driven by word dependencies, and then
learns differentiated contributions from these concepts for following decoding
process. Therefore, our approach captures potential relations among concepts
and discriminatively learns different concepts, so that effectively facilitates
image captioning with inherited information across modalities. Extensive
experiments and their results demonstrate the effectiveness of our approach as
well as each proposed module in this work.
