  We study the problem of detecting human-object interactions (HOI) in static
images, defined as predicting a human and an object bounding box with an
interaction class label that connects them. HOI detection is a fundamental
problem in computer vision as it provides semantic information about the
interactions among the detected objects. We introduce HICO-DET, a new large
benchmark for HOI detection, by augmenting the current HICO classification
benchmark with instance annotations. To solve the task, we propose Human-Object
Region-based Convolutional Neural Networks (HO-RCNN). At the core of our
HO-RCNN is the Interaction Pattern, a novel DNN input that characterizes the
spatial relations between two bounding boxes. Experiments on HICO-DET
demonstrate that our HO-RCNN, by exploiting human-object spatial relations
through Interaction Patterns, significantly improves the performance of HOI
detection over baseline approaches.
