  Visual relationship detection aims to capture interactions between pairs of
objects in images. Relationships between objects and humans represent a
particularly important subset of this problem, with implications for challenges
such as understanding human behaviour, and identifying affordances, amongst
others. In addressing this problem we first construct a large-scale
human-centric visual relationship detection dataset (HCVRD), which provides
many more types of relationship annotation (nearly 10K categories) than the
previous released datasets.
  This large label space better reflects the reality of human-object
interactions, but gives rise to a long-tail distribution problem, which in turn
demands a zero-shot approach to labels appearing only in the test set. This is
the first time this issue has been addressed. We propose a webly-supervised
approach to these problems and demonstrate that the proposed model provides a
strong baseline on our HCVRD dataset.
