  Recent years have seen embodied visual navigation advance in two distinct
directions: (i) in equipping the AI agent to follow natural language
instructions, and (ii) in making the navigable world multimodal, e.g.,
audio-visual navigation. However, the real world is not only multimodal, but
also often complex, and thus in spite of these advances, agents still need to
understand the uncertainty in their actions and seek instructions to navigate.
To this end, we present AVLEN~ -- an interactive agent for
Audio-Visual-Language Embodied Navigation. Similar to audio-visual navigation
tasks, the goal of our embodied agent is to localize an audio event via
navigating the 3D visual world; however, the agent may also seek help from a
human (oracle), where the assistance is provided in free-form natural language.
To realize these abilities, AVLEN uses a multimodal hierarchical reinforcement
learning backbone that learns: (a) high-level policies to choose either
audio-cues for navigation or to query the oracle, and (b) lower-level policies
to select navigation actions based on its audio-visual and language inputs. The
policies are trained via rewarding for the success on the navigation task while
minimizing the number of queries to the oracle. To empirically evaluate AVLEN,
we present experiments on the SoundSpaces framework for semantic audio-visual
navigation tasks. Our results show that equipping the agent to ask for help
leads to a clear improvement in performance, especially in challenging cases,
e.g., when the sound is unheard during training or in the presence of
distractor sounds.
