  Contrastive learning has become a popular technique to pre-train image
encoders, which could be used to build various downstream classification models
in an efficient way. This process requires a large amount of data and
computation resources. Hence, the pre-trained encoders are an important
intellectual property that needs to be carefully protected. It is challenging
to migrate existing watermarking techniques from the classification tasks to
the contrastive learning scenario, as the owner of the encoder lacks the
knowledge of the downstream tasks which will be developed from the encoder in
the future. We propose the \textit{first} watermarking methodology for the
pre-trained encoders. We introduce a task-agnostic loss function to effectively
embed into the encoder a backdoor as the watermark. This backdoor can still
exist in any downstream models transferred from the encoder. Extensive
evaluations over different contrastive learning algorithms, datasets, and
downstream tasks indicate our watermarks exhibit high effectiveness and
robustness against different adversarial operations.
