  Pretrained visual-language models have extensive world knowledge and are
widely used in visual and language navigation (VLN). However, they are not
sensitive to indoor scenarios for VLN tasks. Another challenge for VLN is how
the agent understands the contextual relations between actions on a path and
performs cross-modal alignment sequentially. In this paper, we propose a novel
Prompt-bAsed coNtext- and inDoor-Aware (PANDA) pretraining framework to address
these problems. It performs prompting in two stages. In the indoor-aware stage,
we apply an efficient tuning paradigm to learn deep visual prompts from an
indoor dataset, in order to augment pretrained models with inductive biases
towards indoor environments. This can enable more sample-efficient adaptation
for VLN agents. Furthermore, in the context-aware stage, we design a set of
hard context prompts to capture the sequence-level semantics in the
instruction. They enable further tuning of the pretrained models via
contrastive learning. Experimental results on both R2R and REVERIE show the
superiority of PANDA compared to existing state-of-the-art methods.
