  This paper presents Volumetric Transformer Pose estimator (VTP), the first 3D
volumetric transformer framework for multi-view multi-person 3D human pose
estimation. VTP aggregates features from 2D keypoints in all camera views and
directly learns the spatial relationships in the 3D voxel space in an
end-to-end fashion. The aggregated 3D features are passed through 3D
convolutions before being flattened into sequential embeddings and fed into a
transformer. A residual structure is designed to further improve the
performance. In addition, the sparse Sinkhorn attention is empowered to reduce
the memory cost, which is a major bottleneck for volumetric representations,
while also achieving excellent performance. The output of the transformer is
again concatenated with 3D convolutional features by a residual design. The
proposed VTP framework integrates the high performance of the transformer with
volumetric representations, which can be used as a good alternative to the
convolutional backbones. Experiments on the Shelf, Campus and CMU Panoptic
benchmarks show promising results in terms of both Mean Per Joint Position
Error (MPJPE) and Percentage of Correctly estimated Parts (PCP). Our code will
be available.
