  Human pose estimation (HPE) usually requires large-scale training data to
reach high performance. However, it is rather time-consuming to collect
high-quality and fine-grained annotations for human body. To alleviate this
issue, we revisit HPE and propose a location-free framework without supervision
of keypoint locations. We reformulate the regression-based HPE from the
perspective of classification. Inspired by the CAM-based weakly-supervised
object localization, we observe that the coarse keypoint locations can be
acquired through the part-aware CAMs but unsatisfactory due to the gap between
the fine-grained HPE and the object-level localization. To this end, we propose
a customized transformer framework to mine the fine-grained representation of
human context, equipped with the structural relation to capture subtle
differences among keypoints. Concretely, we design a Multi-scale Spatial-guided
Context Encoder to fully capture the global human context while focusing on the
part-aware regions and a Relation-encoded Pose Prototype Generation module to
encode the structural relations. All these works together for strengthening the
weak supervision from image-level category labels on locations. Our model
achieves competitive performance on three datasets when only supervised at a
category-level and importantly, it can achieve comparable results with
fully-supervised methods with only 25\% location labels on MS-COCO and MPII.
