  We deal with the problem of localized in-video taxonomic human annotation in
the video content moderation domain, where the goal is to identify video
segments that violate granular policies, e.g., community guidelines on an
online video platform. High quality human labeling is critical for enforcement
in content moderation. This is challenging due to the problem of information
overload - raters need to apply a large taxonomy of granular policy violations
with ambiguous definitions, within a limited review duration to relatively long
videos. Our key contribution is a novel human-machine learning (ML)
collaboration framework aimed at maximizing the quality and efficiency of human
decisions in this setting - human labels are used to train segment-level
models, the predictions of which are displayed as "hints" to human raters,
indicating probable regions of the video with specific policy violations. The
human verified/corrected segment labels can help refine the model further,
hence creating a human-ML positive feedback loop. Experiments show improved
human video moderation decision quality, and efficiency through more granular
annotations submitted within a similar review duration, which enable a 5-8% AUC
improvement in the hint generation models.
