  This paper presents Scalable Semantic Transfer (SST), a novel training
paradigm, to explore how to leverage the mutual benefits of the data from
different label domains (i.e. various levels of label granularity) to train a
powerful human parsing network. In practice, two common application scenarios
are addressed, termed universal parsing and dedicated parsing, where the former
aims to learn homogeneous human representations from multiple label domains and
switch predictions by only using different segmentation heads, and the latter
aims to learn a specific domain prediction while distilling the semantic
knowledge from other domains. The proposed SST has the following appealing
benefits: (1) it can capably serve as an effective training scheme to embed
semantic associations of human body parts from multiple label domains into the
human representation learning process; (2) it is an extensible semantic
transfer framework without predetermining the overall relations of multiple
label domains, which allows continuously adding human parsing datasets to
promote the training. (3) the relevant modules are only used for auxiliary
training and can be removed during inference, eliminating the extra reasoning
cost. Experimental results demonstrate SST can effectively achieve promising
universal human parsing performance as well as impressive improvements compared
to its counterparts on three human parsing benchmarks (i.e.,
PASCAL-Person-Part, ATR, and CIHP). Code is available at
https://github.com/yangjie-cv/SST.
