  We present a benchmark for 3D human whole-body pose estimation, which
involves identifying accurate 3D keypoints on the entire human body, including
face, hands, body, and feet. Currently, the lack of a fully annotated and
accurate 3D whole-body dataset results in deep networks being trained
separately on specific body parts, which are combined during inference. Or they
rely on pseudo-groundtruth provided by parametric body models which are not as
accurate as detection based methods. To overcome these issues, we introduce the
Human3.6M 3D WholeBody (H3WB) dataset, which provides whole-body annotations
for the Human3.6M dataset using the COCO Wholebody layout. H3WB comprises 133
whole-body keypoint annotations on 100K images, made possible by our new
multi-view pipeline. We also propose three tasks: i) 3D whole-body pose lifting
from 2D complete whole-body pose, ii) 3D whole-body pose lifting from 2D
incomplete whole-body pose, and iii) 3D whole-body pose estimation from a
single RGB image. Additionally, we report several baselines from popular
methods for these tasks. Furthermore, we also provide automated 3D whole-body
annotations of TotalCapture and experimentally show that when used with H3WB it
helps to improve the performance. Code and dataset is available at
https://github.com/wholebody3d/wholebody3d
