  Code search aims to retrieve semantically relevant code snippets for a given
natural language query. Recently, many approaches employing contrastive
learning have shown promising results on code representation learning and
greatly improved the performance of code search. However, there is still a lot
of room for improvement in using contrastive learning for code search. In this
paper, we propose CoCoSoDa to effectively utilize contrastive learning for code
search via two key factors in contrastive learning: data augmentation and
negative samples. Specifically, soft data augmentation is to dynamically
masking or replacing some tokens with their types for input sequences to
generate positive samples. Momentum mechanism is used to generate large and
consistent representations of negative samples in a mini-batch through
maintaining a queue and a momentum encoder. In addition, multimodal contrastive
learning is used to pull together representations of code-query pairs and push
apart the unpaired code snippets and queries. We conduct extensive experiments
to evaluate the effectiveness of our approach on a large-scale dataset with six
programming languages. Experimental results show that: (1) CoCoSoDa outperforms
14 baselines and especially exceeds CodeBERT, GraphCodeBERT, and UniXcoder by
13.3%, 10.5%, and 5.9% on average MRR scores, respectively. (2) The ablation
studies show the effectiveness of each component of our approach. (3) We adapt
our techniques to several different pre-trained models such as RoBERTa,
CodeBERT, and GraphCodeBERT and observe a significant boost in their
performance in code search. (4) Our model performs robustly under different
hyper-parameters. Furthermore, we perform qualitative and quantitative analyses
to explore reasons behind the good performance of our model.
