  We target at the task of weakly-supervised video object grounding (WSVOG),
where only video-sentence annotations are available during model learning. It
aims to localize objects described in the sentence to visual regions in the
video, which is a fundamental capability needed in pattern analysis and machine
learning. Despite the recent progress, existing methods all suffer from the
severe problem of spurious association, which will harm the grounding
performance. In this paper, we start from the definition of WSVOG and pinpoint
the spurious association from two aspects: (1) the association itself is not
object-relevant but extremely ambiguous due to weak supervision, and (2) the
association is unavoidably confounded by the observational bias when taking the
statistics-based matching strategy in existing methods. With this in mind, we
design a unified causal framework to learn the deconfounded object-relevant
association for more accurate and robust video object grounding. Specifically,
we learn the object-relevant association by causal intervention from the
perspective of video data generation process. To overcome the problems of
lacking fine-grained supervision in terms of intervention, we propose a novel
spatial-temporal adversarial contrastive learning paradigm. To further remove
the accompanying confounding effect within the object-relevant association, we
pursue the true causality by conducting causal intervention via backdoor
adjustment. Finally, the deconfounded object-relevant association is learned
and optimized under a unified causal framework in an end-to-end manner.
Extensive experiments on both IID and OOD testing sets of three benchmarks
demonstrate its accurate and robust grounding performance against
state-of-the-arts.
