  Phrase Grounding aims to detect and localize objects in images that are
referred to and are queried by natural language phrases. Phrase grounding finds
applications in tasks such as Visual Dialog, Visual Search and Image-text
co-reference resolution. In this paper, we present a framework that leverages
information such as phrase category, relationships among neighboring phrases in
a sentence and context to improve the performance of phrase grounding systems.
We propose three modules: Proposal Indexing Network(PIN); Inter-phrase
Regression Network(IRN) and Proposal Ranking Network(PRN) each of which analyze
the region proposals of an image at increasing levels of detail by
incorporating the above information. Also, in the absence of ground-truth
spatial locations of the phrases(weakly-supervised), we propose knowledge
transfer mechanisms that leverages the framework of PIN module. We demonstrate
the effectiveness of our approach on the Flickr 30k Entities and ReferItGame
datasets, for which we achieve improvements over state-of-the-art approaches in
both supervised and weakly-supervised variants.
