  Multimodal contrastive learning (MCL) has shown remarkable advances in
zero-shot classification by learning from millions of image-caption pairs
crawled from the Internet. However, this reliance poses privacy risks, as
hackers may unauthorizedly exploit image-text data for model training,
potentially including personal and privacy-sensitive information. Recent works
propose generating unlearnable examples by adding imperceptible perturbations
to training images to build shortcuts for protection. However, they are
designed for unimodal classification, which remains largely unexplored in MCL.
We first explore this context by evaluating the performance of existing methods
on image-caption pairs, and they do not generalize effectively to multimodal
data and exhibit limited impact to build shortcuts due to the lack of labels
and the dispersion of pairs in MCL. In this paper, we propose Multi-step Error
Minimization (MEM), a novel optimization process for generating multimodal
unlearnable examples. It extends the Error-Minimization (EM) framework to
optimize both image noise and an additional text trigger, thereby enlarging the
optimized space and effectively misleading the model to learn the shortcut
between the noise features and the text trigger. Specifically, we adopt
projected gradient descent to solve the noise minimization problem and use
HotFlip to approximate the gradient and replace words to find the optimal text
trigger. Extensive experiments demonstrate the effectiveness of MEM, with
post-protection retrieval results nearly half of random guessing, and its high
transferability across different models. Our code is available on the
https://github.com/thinwayliu/Multimodal-Unlearnable-Examples
