  In this work, we study the problem of Embodied Referring Expression
Grounding, where an agent needs to navigate in a previously unseen environment
and localize a remote object described by a concise high-level natural language
instruction. When facing such a situation, a human tends to imagine what the
destination may look like and to explore the environment based on prior
knowledge of the environmental layout, such as the fact that a bathroom is more
likely to be found near a bedroom than a kitchen. We have designed an
autonomous agent called Layout-aware Dreamer (LAD), including two novel
modules, that is, the Layout Learner and the Goal Dreamer to mimic this
cognitive decision process. The Layout Learner learns to infer the room
category distribution of neighboring unexplored areas along the path for coarse
layout estimation, which effectively introduces layout common sense of
room-to-room transitions to our agent. To learn an effective exploration of the
environment, the Goal Dreamer imagines the destination beforehand. Our agent
achieves new state-of-the-art performance on the public leaderboard of the
REVERIE dataset in challenging unseen test environments with improvement in
navigation success (SR) by 4.02% and remote grounding success (RGS) by 3.43%
compared to the previous state-of-the-art. The code is released at
https://github.com/zehao-wang/LAD
