  Human parsing and pose estimation have recently received considerable
interest due to their substantial application potentials. However, the existing
datasets have limited numbers of images and annotations and lack a variety of
human appearances and coverage of challenging cases in unconstrained
environments. In this paper, we introduce a new benchmark named "Look into
Person (LIP)" that provides a significant advancement in terms of scalability,
diversity, and difficulty, which are crucial for future developments in
human-centric analysis. This comprehensive dataset contains over 50,000
elaborately annotated images with 19 semantic part labels and 16 body joints,
which are captured from a broad range of viewpoints, occlusions, and background
complexities. Using these rich annotations, we perform detailed analyses of the
leading human parsing and pose estimation approaches, thereby obtaining
insights into the successes and failures of these methods. To further explore
and take advantage of the semantic correlation of these two tasks, we propose a
novel joint human parsing and pose estimation network to explore efficient
context modeling, which can simultaneously predict parsing and pose with
extremely high quality. Furthermore, we simplify the network to solve human
parsing by exploring a novel self-supervised structure-sensitive learning
approach, which imposes human pose structures into the parsing results without
resorting to extra supervision. The dataset, code and models are available at
http://www.sysu-hcp.net/lip/.
