  Rapid progress has been witnessed for human-object interaction (HOI)
recognition, but most existing models are confined to single-stage reasoning
pipelines. Considering the intrinsic complexity of the task, we introduce a
cascade architecture for a multi-stage, coarse-to-fine HOI understanding. At
each stage, an instance localization network progressively refines HOI
proposals and feeds them into an interaction recognition network. Each of the
two networks is also connected to its predecessor at the previous stage,
enabling cross-stage information propagation. The interaction recognition
network has two crucial parts: a relation ranking module for high-quality HOI
proposal selection and a triple-stream classifier for relation prediction. With
our carefully-designed human-centric relation features, these two modules work
collaboratively towards effective interaction understanding. Further beyond
relation detection on a bounding-box level, we make our framework flexible to
perform fine-grained pixel-wise relation segmentation; this provides a new
glimpse into better relation modeling. Our approach reached the $1^{st}$ place
in the ICCV2019 Person in Context Challenge, on both relation detection and
segmentation tasks. It also shows promising results on V-COCO.
