  Prior highly-tuned human parsing models tend to fit towards each dataset in a
specific domain or with discrepant label granularity, and can hardly be adapted
to other human parsing tasks without extensive re-training. In this paper, we
aim to learn a single universal human parsing model that can tackle all kinds
of human parsing needs by unifying label annotations from different domains or
at various levels of granularity. This poses many fundamental learning
challenges, e.g. discovering underlying semantic structures among different
label granularity, performing proper transfer learning across different image
domains, and identifying and utilizing label redundancies across related tasks.
  To address these challenges, we propose a new universal human parsing agent,
named "Graphonomy", which incorporates hierarchical graph transfer learning
upon the conventional parsing network to encode the underlying label semantic
structures and propagate relevant semantic information. In particular,
Graphonomy first learns and propagates compact high-level graph representation
among the labels within one dataset via Intra-Graph Reasoning, and then
transfers semantic information across multiple datasets via Inter-Graph
Transfer. Various graph transfer dependencies (\eg, similarity, linguistic
knowledge) between different datasets are analyzed and encoded to enhance graph
transfer capability. By distilling universal semantic graph representation to
each specific task, Graphonomy is able to predict all levels of parsing labels
in one system without piling up the complexity. Experimental results show
Graphonomy effectively achieves the state-of-the-art results on three human
parsing benchmarks as well as advantageous universal human parsing performance.
