  Vision-and-Language Navigation (VLN) task aims to enable AI agents to
accurately understand and follow natural language instructions to navigate
through real-world environments, ultimately reaching specific target locations.
We recognise a promising opportunity to extend VLN to a comparable navigation
task that holds substantial significance in our daily lives, albeit within the
virtual realm: navigating websites on the Internet. This paper proposes a new
task named Vision-and-Language Navigation on Websites (WebVLN), where we use
question-based instructions to train an agent, emulating how users naturally
browse websites. Unlike the existing VLN task that only pays attention to
vision and instruction (language), the WebVLN agent further considers
underlying web-specific content like HTML, which could not be seen on the
rendered web pages yet contains rich visual and textual information. Toward
this goal, we contribute a dataset, WebVLN-v1, and introduce a novel approach
called Website-aware VLN Network (WebVLN-Net), which is built upon the
foundation of state-of-the-art VLN techniques. Experimental results show that
WebVLN-Net outperforms current VLN and web-related navigation methods. We
believe that the introduction of the new WebVLN task and its dataset will
establish a new dimension within the VLN domain and contribute to the broader
vision-and-language research community. The code is available at:
https://github.com/WebVLN/WebVLN.
