  The key to integrating visual language tasks is to establish a good alignment
strategy. Recently, visual semantic representation has achieved fine-grained
visual understanding by dividing grids or image patches. However, the
coarse-grained semantic interactions in image space should not be ignored,
which hinders the extraction of complex contextual semantic relations at the
scene boundaries. This paper proposes superpixels as comprehensive and robust
visual primitives, which mine coarse-grained semantic interactions by
clustering perceptually similar pixels, speeding up the subsequent processing
of primitives. To capture superpixel-level semantic features, we propose a
Multiscale Difference Graph Convolutional Network (MDGCN). It allows parsing
the entire image as a fine-to-coarse visual hierarchy. To reason actual
semantic relations, we reduce potential noise interference by aggregating
difference information between adjacent graph nodes. Finally, we propose a
multi-level fusion rule in a bottom-up manner to avoid understanding deviation
by mining complementary spatial information at different levels. Experiments
show that the proposed method can effectively promote the learning of multiple
downstream tasks. Encouragingly, our method outperforms previous methods on all
metrics. Our code will be released upon publication.
