  Human object interaction (HOI) detection plays a crucial role in
human-centric scene understanding and serves as a fundamental building-block
for many vision tasks. One generalizable and scalable strategy for HOI
detection is to use weak supervision, learning from image-level annotations
only. This is inherently challenging due to ambiguous human-object
associations, large search space of detecting HOIs and highly noisy training
signal. A promising strategy to address those challenges is to exploit
knowledge from large-scale pretrained models (e.g., CLIP), but a direct
knowledge distillation strategy~\citep{liao2022gen} does not perform well on
the weakly-supervised setting. In contrast, we develop a CLIP-guided HOI
representation capable of incorporating the prior knowledge at both image level
and HOI instance level, and adopt a self-taught mechanism to prune incorrect
human-object associations. Experimental results on HICO-DET and V-COCO show
that our method outperforms the previous works by a sizable margin, showing the
efficacy of our HOI representation.
