  Recently, image-text matching has attracted more and more attention from
academia and industry, which is fundamental to understanding the latent
correspondence across visual and textual modalities. However, most existing
methods implicitly assume the training pairs are well-aligned while ignoring
the ubiquitous annotation noise, a.k.a noisy correspondence (NC), thereby
inevitably leading to a performance drop. Although some methods attempt to
address such noise, they still face two challenging problems: excessive
memorizing/overfitting and unreliable correction for NC, especially under high
noise. To address the two problems, we propose a generalized Cross-modal Robust
Complementary Learning framework (CRCL), which benefits from a novel Active
Complementary Loss (ACL) and an efficient Self-refining Correspondence
Correction (SCC) to improve the robustness of existing methods. Specifically,
ACL exploits active and complementary learning losses to reduce the risk of
providing erroneous supervision, leading to theoretically and experimentally
demonstrated robustness against NC. SCC utilizes multiple self-refining
processes with momentum correction to enlarge the receptive field for
correcting correspondences, thereby alleviating error accumulation and
achieving accurate and stable corrections. We carry out extensive experiments
on three image-text benchmarks, i.e., Flickr30K, MS-COCO, and CC152K, to verify
the superior robustness of our CRCL against synthetic and real-world noisy
correspondences.
