  Most existing works in vision-and-language navigation (VLN) focus on either
discrete or continuous environments, training agents that cannot generalize
across the two. The fundamental difference between the two setups is that
discrete navigation assumes prior knowledge of the connectivity graph of the
environment, so that the agent can effectively transfer the problem of
navigation with low-level controls to jumping from node to node with high-level
actions by grounding to an image of a navigable direction. To bridge the
discrete-to-continuous gap, we propose a predictor to generate a set of
candidate waypoints during navigation, so that agents designed with high-level
actions can be transferred to and trained in continuous environments. We refine
the connectivity graph of Matterport3D to fit the continuous
Habitat-Matterport3D, and train the waypoints predictor with the refined graphs
to produce accessible waypoints at each time step. Moreover, we demonstrate
that the predicted waypoints can be augmented during training to diversify the
views and paths, and therefore enhance agent's generalization ability. Through
extensive experiments we show that agents navigating in continuous environments
with predicted waypoints perform significantly better than agents using
low-level actions, which reduces the absolute discrete-to-continuous gap by
11.76% Success Weighted by Path Length (SPL) for the Cross-Modal Matching Agent
and 18.24% SPL for the Recurrent VLN-BERT. Our agents, trained with a simple
imitation learning objective, outperform previous methods by a large margin,
achieving new state-of-the-art results on the testing environments of the
R2R-CE and the RxR-CE datasets.
