  Multi-agent embodied tasks have recently been studied in complex indoor
visual environments. Collaboration among multiple agents can improve work
efficiency and has significant practical value. However, most of the existing
research focuses on homogeneous multi-agent tasks. Compared with homogeneous
agents, heterogeneous agents can leverage their different capabilities to
allocate corresponding sub-tasks and cooperate to complete complex tasks.
Heterogeneous multi-agent tasks are common in real-world scenarios, and the
collaboration strategy among heterogeneous agents is a challenging and
important problem to be solved. To study collaboration among heterogeneous
agents, we propose the heterogeneous multi-agent tidying-up task, in which
multiple heterogeneous agents with different capabilities collaborate with each
other to detect misplaced objects and place them in reasonable locations. This
is a demanding task since it requires agents to make the best use of their
different capabilities to conduct reasonable task planning and complete the
whole task. To solve this task, we build a heterogeneous multi-agent tidying-up
benchmark dataset in a large number of houses with multiple rooms based on
ProcTHOR-10K. We propose the hierarchical decision model based on misplaced
object detection, reasonable receptacle prediction, as well as the
handshake-based group communication mechanism. Extensive experiments are
conducted to demonstrate the effectiveness of the proposed model. The project's
website and videos of experiments can be found at https://hetercol.github.io/.
