  Zero-shot composed image retrieval (ZS-CIR), which aims to retrieve a target
image based on textual modifications to a reference image without triplet
labeling, has gained more and more attention. Current ZS-CIR research mainly
relies on two unlabeled pre-trained models: the vision-language model, e.g.,
CLIP, and the Pic2Word/textual inversion model. However, the pre-trained models
and CIR tasks have substantial discrepancies, where the pre-trained models
learn the similarities between vision and language but CIR aims to learn the
modifications of the image guided by text. In this paper, we introduce a novel
unlabeled and pre-trained masked tuning approach to reduce the gap between the
pre-trained model and the downstream CIR task. We first reformulate the
pre-trained vision-language contrastive learning as the CIR task, where we
randomly mask input image patches to generate $\langle$masked image, text,
image$\rangle$ triple from an image-text pair. Then, we propose a masked
tuning, which uses the text and the masked image to learn the modifications of
the original image. With such a simple design, it can learn to capture
fine-grained text-guided modifications. Extensive experimental results
demonstrate the significant superiority of our approach over the baseline
models on three ZS-CIR datasets, including FashionIQ, CIRR, and CIRCO.
