  The ability to estimate 3D human body pose and movement, also known as human
pose estimation (HPE), enables many applications for home-based health
monitoring, such as remote rehabilitation training. Several possible solutions
have emerged using sensors ranging from RGB cameras, depth sensors,
millimeter-Wave (mmWave) radars, and wearable inertial sensors. Despite
previous efforts on datasets and benchmarks for HPE, few dataset exploits
multiple modalities and focuses on home-based health monitoring. To bridge the
gap, we present mRI, a multi-modal 3D human pose estimation dataset with
mmWave, RGB-D, and Inertial Sensors. Our dataset consists of over 160k
synchronized frames from 20 subjects performing rehabilitation exercises and
supports the benchmarks of HPE and action detection. We perform extensive
experiments using our dataset and delineate the strength of each modality. We
hope that the release of mRI can catalyze the research in pose estimation,
multi-modal learning, and action understanding, and more importantly facilitate
the applications of home-based health monitoring.
