  We present a new method, PARsing And visual GrOuNding (ParaGon), for
grounding natural language in object placement tasks. Natural language
generally describes objects and spatial relations with compositionality and
ambiguity, two major obstacles to effective language grounding. For
compositionality, ParaGon parses a language instruction into an object-centric
graph representation to ground objects individually. For ambiguity, ParaGon
uses a novel particle-based graph neural network to reason about object
placements with uncertainty. Essentially, ParaGon integrates a parsing
algorithm into a probabilistic, data-driven learning framework. It is fully
differentiable and trained end-to-end from data for robustness against complex,
ambiguous language input.
