  Audio-visual navigation task requires an agent to find a sound source in a
realistic, unmapped 3D environment by utilizing egocentric audio-visual
observations. Existing audio-visual navigation works assume a clean environment
that solely contains the target sound, which, however, would not be suitable in
most real-world applications due to the unexpected sound noise or intentional
interference. In this work, we design an acoustically complex environment in
which, besides the target sound, there exists a sound attacker playing a
zero-sum game with the agent. More specifically, the attacker can move and
change the volume and category of the sound to make the agent suffer from
finding the sounding object while the agent tries to dodge the attack and
navigate to the goal under the intervention. Under certain constraints to the
attacker, we can improve the robustness of the agent towards unexpected sound
attacks in audio-visual navigation. For better convergence, we develop a joint
training mechanism by employing the property of a centralized critic with
decentralized actors. Experiments on two real-world 3D scan datasets, Replica,
and Matterport3D, verify the effectiveness and the robustness of the agent
trained under our designed environment when transferred to the clean
environment or the one containing sound attackers with random policy. Project:
\url{https://yyf17.github.io/SAAVN}.
