  Understanding interactions between humans and objects is one of the
fundamental problems in visual classification and an essential step towards
detailed scene understanding. Human-object interaction (HOI) detection strives
to localize both the human and an object as well as the identification of
complex interactions between them. Most existing HOI detection approaches are
instance-centric where interactions between all possible human-object pairs are
predicted based on appearance features and coarse spatial information. We argue
that appearance features alone are insufficient to capture complex human-object
interactions. In this paper, we therefore propose a novel fully-convolutional
approach that directly detects the interactions between human-object pairs. Our
network predicts interaction points, which directly localize and classify the
inter-action. Paired with the densely predicted interaction vectors, the
interactions are associated with human and object detections to obtain final
predictions. To the best of our knowledge, we are the first to propose an
approach where HOI detection is posed as a keypoint detection and grouping
problem. Experiments are performed on two popular benchmarks: V-COCO and
HICO-DET. Our approach sets a new state-of-the-art on both datasets. Code is
available at https://github.com/vaesl/IP-Net.
