  Weakly supervised referring expression grounding aims at localizing the
referential object in an image according to the linguistic query, where the
mapping between the referential object and query is unknown in the training
stage. To address this problem, we propose a novel end-to-end adaptive
reconstruction network (ARN). It builds the correspondence between image region
proposal and query in an adaptive manner: adaptive grounding and collaborative
reconstruction. Specifically, we first extract the subject, location and
context features to represent the proposals and the query respectively. Then,
we design the adaptive grounding module to compute the matching score between
each proposal and query by a hierarchical attention model. Finally, based on
attention score and proposal features, we reconstruct the input query with a
collaborative loss of language reconstruction loss, adaptive reconstruction
loss, and attribute classification loss. This adaptive mechanism helps our
model to alleviate the variance of different referring expressions. Experiments
on four large-scale datasets show ARN outperforms existing state-of-the-art
methods by a large margin. Qualitative results demonstrate that the proposed
ARN can better handle the situation where multiple objects of a particular
category situated together.
