  We tackle the problem of cooperative visual exploration where multiple agents
need to jointly explore unseen regions as fast as possible based on visual
signals. Classical planning-based methods often suffer from expensive
computation overhead at each step and a limited expressiveness of complex
cooperation strategy. By contrast, reinforcement learning (RL) has recently
become a popular paradigm for tackling this challenge due to its modeling
capability of arbitrarily complex strategies and minimal inference overhead. In
this paper, we extend the state-of-the-art single-agent visual navigation
method, Active Neural SLAM (ANS), to the multi-agent setting by introducing a
novel RL-based planning module, Multi-agent Spatial Planner (MSP).MSP leverages
a transformer-based architecture, Spatial-TeamFormer, which effectively
captures spatial relations and intra-agent interactions via hierarchical
spatial self-attentions. In addition, we also implement a few multi-agent
enhancements to process local information from each agent for an aligned
spatial representation and more precise planning. Finally, we perform policy
distillation to extract a meta policy to significantly improve the
generalization capability of final policy. We call this overall solution,
Multi-Agent Active Neural SLAM (MAANS). MAANS substantially outperforms
classical planning-based baselines for the first time in a photo-realistic 3D
simulator, Habitat. Code and videos can be found at
https://sites.google.com/view/maans.
