  Fully supervised human mesh recovery methods are data-hungry and have poor
generalizability due to the limited availability and diversity of 3D-annotated
benchmark datasets. Recent progress in self-supervised human mesh recovery has
been made using synthetic-data-driven training paradigms where the model is
trained from synthetic paired 2D representation (e.g., 2D keypoints and
segmentation masks) and 3D mesh. However, on synthetic dense correspondence
maps (i.e., IUV) few have been explored since the domain gap between synthetic
training data and real testing data is hard to address for 2D dense
representation. To alleviate this domain gap on IUV, we propose
cross-representation alignment utilizing the complementary information from the
robust but sparse representation (2D keypoints). Specifically, the alignment
errors between initial mesh estimation and both 2D representations are
forwarded into regressor and dynamically corrected in the following mesh
regression. This adaptive cross-representation alignment explicitly learns from
the deviations and captures complementary information: robustness from sparse
representation and richness from dense representation. We conduct extensive
experiments on multiple standard benchmark datasets and demonstrate competitive
results, helping take a step towards reducing the annotation effort needed to
produce state-of-the-art models in human mesh estimation.
