  Vision-Language Navigation (VLN) requires the agent to follow language
instructions to reach a target position. A key factor for successful navigation
is to align the landmarks implied in the instruction with diverse visual
observations. However, previous VLN agents fail to perform accurate modality
alignment especially in unexplored scenes, since they learn from limited
navigation data and lack sufficient open-world alignment knowledge. In this
work, we propose a new VLN paradigm, called COrrectable LaNdmark DiScOvery via
Large ModEls (CONSOLE). In CONSOLE, we cast VLN as an open-world sequential
landmark discovery problem, by introducing a novel correctable landmark
discovery scheme based on two large models ChatGPT and CLIP. Specifically, we
use ChatGPT to provide rich open-world landmark cooccurrence commonsense, and
conduct CLIP-driven landmark discovery based on these commonsense priors. To
mitigate the noise in the priors due to the lack of visual constraints, we
introduce a learnable cooccurrence scoring module, which corrects the
importance of each cooccurrence according to actual observations for accurate
landmark discovery. We further design an observation enhancement strategy for
an elegant combination of our framework with different VLN agents, where we
utilize the corrected landmark features to obtain enhanced observation features
for action decision. Extensive experimental results on multiple popular VLN
benchmarks (R2R, REVERIE, R4R, RxR) show the significant superiority of CONSOLE
over strong baselines. Especially, our CONSOLE establishes the new
state-of-the-art results on R2R and R4R in unseen scenarios. Code is available
at https://github.com/expectorlin/CONSOLE.
