  Human-Object Interaction (HOI) detection is a task to localize humans and
objects in an image and predict the interactions in human-object pairs. In
real-world scenarios, HOI detection models need systematic generalization,
i.e., generalization to novel combinations of objects and interactions, because
the train data are expected to cover a limited portion of all possible
combinations. To evaluate the systematic generalization performance of HOI
detection models, we created two new sets of HOI detection data splits named
HICO-DET-SG and V-COCO-SG based on the HICO-DET and V-COCO datasets,
respectively. When evaluated on the new data splits, HOI detection models with
various characteristics performed much more poorly than when evaluated on the
original splits. This shows that systematic generalization is a challenging
goal in HOI detection. By analyzing the evaluation results, we also gain
insights for improving the systematic generalization performance and identify
four possible future research directions. We hope that our new data splits and
presented analysis will encourage further research on systematic generalization
in HOI detection.
