  In this paper, we propose the problem of collaborative perception, where
robots can combine their local observations with those of neighboring agents in
a learnable way to improve accuracy on a perception task. Unlike existing work
in robotics and multi-agent reinforcement learning, we formulate the problem as
one where learned information must be shared across a set of agents in a
bandwidth-sensitive manner to optimize for scene understanding tasks such as
semantic segmentation. Inspired by networking communication protocols, we
propose a multi-stage handshake communication mechanism where the neural
network can learn to compress relevant information needed for each stage.
Specifically, a target agent with degraded sensor data sends a compressed
request, the other agents respond with matching scores, and the target agent
determines who to connect with (i.e., receive information from). We
additionally develop the AirSim-CP dataset and metrics based on the AirSim
simulator where a group of aerial robots perceive diverse landscapes, such as
roads, grasslands, buildings, etc. We show that for the semantic segmentation
task, our handshake communication method significantly improves accuracy by
approximately 20% over decentralized baselines, and is comparable to
centralized ones using a quarter of the bandwidth.
