  Traditional approaches for complementary product recommendations rely on
behavioral and non-visual data such as customer co-views or co-buys. However,
certain domains such as fashion are primarily visual. We propose a framework
that harnesses visual cues in an unsupervised manner to learn the distribution
of co-occurring complementary items in real world images. Our model learns a
non-linear transformation between the two manifolds of source and target
complementary item categories (e.g., tops and bottoms in outfits). Given a
large dataset of images containing instances of co-occurring object categories,
we train a generative transformer network directly on the feature
representation space by casting it as an adversarial optimization problem. Such
a conditional generative model can produce multiple novel samples of
complementary items (in the feature space) for a given query item. The final
recommendations are selected from the closest real world examples to the
synthesized complementary features. We apply our framework to the task of
recommending complementary tops for a given bottom clothing item. The
recommendations made by our system are diverse, and are favored by human
experts over the baseline approaches.
