  We present a new zero-shot dense retrieval (ZeroDR) method, COCO-DR, to
improve the generalization ability of dense retrieval by combating the
distribution shifts between source training tasks and target scenarios. To
mitigate the impact of document differences, COCO-DR continues pretraining the
language model on the target corpora to adapt the model to target distributions
via COtinuous COtrastive learning. To prepare for unseen target queries,
COCO-DR leverages implicit Distributionally Robust Optimization (iDRO) to
reweight samples from different source query clusters for improving model
robustness over rare queries during fine-tuning. COCO-DR achieves superior
average performance on BEIR, the zero-shot retrieval benchmark. At BERT Base
scale, COCO-DR Base outperforms other ZeroDR models with 60x larger size. At
BERT Large scale, COCO-DR Large outperforms the giant GPT-3 embedding model
which has 500x more parameters. Our analysis show the correlation between
COCO-DR's effectiveness in combating distribution shifts and improving
zero-shot accuracy. Our code and model can be found at
\url{https://github.com/OpenMatch/COCO-DR}.
