  Existing lifting networks for regressing 3D human poses from 2D single-view
poses are typically constructed with linear layers based on graph-structured
representation learning. In sharp contrast to them, this paper presents Grid
Convolution (GridConv), mimicking the wisdom of regular convolution operations
in image space. GridConv is based on a novel Semantic Grid Transformation (SGT)
which leverages a binary assignment matrix to map the irregular
graph-structured human pose onto a regular weave-like grid pose representation
joint by joint, enabling layer-wise feature learning with GridConv operations.
We provide two ways to implement SGT, including handcrafted and learnable
designs. Surprisingly, both designs turn out to achieve promising results and
the learnable one is better, demonstrating the great potential of this new
lifting representation learning formulation. To improve the ability of GridConv
to encode contextual cues, we introduce an attention module over the
convolutional kernel, making grid convolution operations input-dependent,
spatial-aware and grid-specific. We show that our fully convolutional grid
lifting network outperforms state-of-the-art methods with noticeable margins
under (1) conventional evaluation on Human3.6M and (2) cross-evaluation on
MPI-INF-3DHP. Code is available at https://github.com/OSVAI/GridConv
