  Cross-modal Retrieval methods build similarity relations between vision and
language modalities by jointly learning a common representation space. However,
the predictions are often unreliable due to the Aleatoric uncertainty, which is
induced by low-quality data, e.g., corrupt images, fast-paced videos, and
non-detailed texts. In this paper, we propose a novel Prototype-based Aleatoric
Uncertainty Quantification (PAU) framework to provide trustworthy predictions
by quantifying the uncertainty arisen from the inherent data ambiguity.
Concretely, we first construct a set of various learnable prototypes for each
modality to represent the entire semantics subspace. Then Dempster-Shafer
Theory and Subjective Logic Theory are utilized to build an evidential
theoretical framework by associating evidence with Dirichlet Distribution
parameters. The PAU model induces accurate uncertainty and reliable predictions
for cross-modal retrieval. Extensive experiments are performed on four major
benchmark datasets of MSR-VTT, MSVD, DiDeMo, and MS-COCO, demonstrating the
effectiveness of our method. The code is accessible at
https://github.com/leolee99/PAU.
