  We have witnessed significant progress in human-object interaction (HOI)
detection. The reliance on mAP (mean Average Precision) scores as a summary
metric, however, does not provide sufficient insight into the nuances of model
performance (e.g., why one model is better than another), which can hinder
further innovation in this field. To address this issue, in this paper, we
introduce a diagnosis toolbox to provide detailed quantitative break-down
analysis of HOI detection models, inspired by the success of object detection
diagnosis toolboxes. We first conduct holistic investigations in the pipeline
of HOI detection. By defining a set of errors and the oracles to fix each of
them, we can have a quantitative analysis of the significance of different
errors according to the mAP improvement obtained from fixing each error. We
then delve into two sub-tasks of HOI detection: human-object pair detection and
interaction classification, respectively. For the first detection task, we
compute the coverage of ground-truth human-object pairs as well as the
noisiness level in the detection results. For the second classification task,
we measure a model's performance of differentiating positive and negative
detection results and also classifying the actual interactions when the
human-object pairs are correctly detected. We analyze eight state-of-the-art
HOI detection models and provide valuable diagnosis insights to foster future
research. For instance, our diagnosis shows that state-of-the-art model RLIPv2
outperforms others mainly because it significantly improves the multi-label
interaction classification accuracy. Our toolbox is applicable for different
methods across different datasets and available at
https://github.com/neu-vi/Diag-HOI.
