  Enlightened by the big success of pre-training in natural language
processing, pre-trained models for programming languages have been widely used
to promote code intelligence in recent years. In particular, BERT has been used
for bug localization tasks and impressive results have been obtained. However,
these BERT-based bug localization techniques suffer from two issues. First, the
pre-trained BERT model on source code does not adequately capture the deep
semantics of program code. Second, the overall bug localization models neglect
the necessity of large-scale negative samples in contrastive learning for
representations of changesets and ignore the lexical similarity between bug
reports and changesets during similarity estimation. We address these two
issues by 1) proposing a novel directed, multiple-label code graph
representation named Semantic Flow Graph (SFG), which compactly and adequately
captures code semantics, 2) designing and training SemanticCodeBERT based on
SFG, and 3) designing a novel Hierarchical Momentum Contrastive Bug
Localization technique (HMCBL). Evaluation results show that our method
achieves state-of-the-art performance in bug localization.
