  Vision-and-Language Navigation (VLN) requires an agent to follow
natural-language instructions, explore the given environments, and reach the
desired target locations. These step-by-step navigational instructions are
crucial when the agent is navigating new environments about which it has no
prior knowledge. Most recent works that study VLN observe a significant
performance drop when tested on unseen environments (i.e., environments not
used in training), indicating that the neural agent models are highly biased
towards training environments. Although this issue is considered as one of the
major challenges in VLN research, it is still under-studied and needs a clearer
explanation. In this work, we design novel diagnosis experiments via
environment re-splitting and feature replacement, looking into possible reasons
for this environment bias. We observe that neither the language nor the
underlying navigational graph, but the low-level visual appearance conveyed by
ResNet features directly affects the agent model and contributes to this
environment bias in results. According to this observation, we explore several
kinds of semantic representations that contain less low-level visual
information, hence the agent learned with these features could be better
generalized to unseen testing environments. Without modifying the baseline
agent model and its training method, our explored semantic features
significantly decrease the performance gaps between seen and unseen on multiple
datasets (i.e. R2R, R4R, and CVDN) and achieve competitive unseen results to
previous state-of-the-art models. Our code and features are available at:
https://github.com/zhangybzbo/EnvBiasVLN
