  Learning the compatibility between fashion items across categories is a key
task in fashion analysis, which can decode the secret of clothing matching. The
main idea of this task is to map items into a latent style space where
compatible items stay close. Previous works try to build such a transformation
by minimizing the distances between annotated compatible items, which require
massive item-level supervision. However, these annotated data are expensive to
obtain and hard to cover the numerous items with various styles in real
applications. In such cases, these supervised methods fail to achieve
satisfactory performances. In this work, we propose a semi-supervised method to
learn the compatibility across categories. We observe that the distributions of
different categories have intrinsic similar structures. Accordingly, the better
distributions align, the closer compatible items across these categories
become. To achieve the alignment, we minimize the distances between
distributions with unsupervised adversarial learning, and also the distances
between some annotated compatible items which play the role of anchor points to
help align. Experimental results on two real-world datasets demonstrate the
effectiveness of our method.
