  3D human pose estimation in outdoor environments has garnered increasing
attention recently. However, prevalent 3D human pose datasets pertaining to
outdoor scenes lack diversity, as they predominantly utilize only one type of
modality (RGB image or pointcloud), and often feature only one individual
within each scene. This limited scope of dataset infrastructure considerably
hinders the variability of available data. In this article, we propose
Human-M3, an outdoor multi-modal multi-view multi-person human pose database
which includes not only multi-view RGB videos of outdoor scenes but also
corresponding pointclouds. In order to obtain accurate human poses, we propose
an algorithm based on multi-modal data input to generate ground truth
annotation. This benefits from robust pointcloud detection and tracking, which
solves the problem of inaccurate human localization and matching ambiguity that
may exist in previous multi-view RGB videos in outdoor multi-person scenes, and
generates reliable ground truth annotations. Evaluation of multiple different
modalities algorithms has shown that this database is challenging and suitable
for future research. Furthermore, we propose a 3D human pose estimation
algorithm based on multi-modal data input, which demonstrates the advantages of
multi-modal data input for 3D human pose estimation. Code and data will be
released on https://github.com/soullessrobot/Human-M3-Dataset.
