  Non-parametric mesh reconstruction has recently shown significant progress in
3D hand and body applications. In these methods, mesh vertices and edges are
visible to neural networks, enabling the possibility to establish a direct
mapping between 2D image pixels and 3D mesh vertices. In this paper, we seek to
establish and exploit this mapping with a simple and compact architecture. The
network is designed with these considerations: 1) aggregating both local 2D
image features from the encoder and 3D geometric features captured in the mesh
decoder; 2) decoding coarse-to-fine meshes along the decoding layers to make
the best use of the hierarchical multi-scale information. Specifically, we
propose an end-to-end pipeline for hand mesh recovery tasks which consists of
three phases: a 2D feature extractor constructing multi-scale feature maps, a
feature mapping module transforming local 2D image features to 3D vertex
features via 3D-to-2D projection, and a mesh decoder combining the graph
convolution and self-attention to reconstruct mesh. The decoder aggregate both
local image features in pixels and geometric features in vertices. It also
regresses the mesh vertices in a coarse-to-fine manner, which can leverage
multi-scale information. By exploiting the local connection and designing the
mesh decoder, Our approach achieves state-of-the-art for hand mesh
reconstruction on the public FreiHAND dataset.
