  Abstract syntax trees (ASTs) play a crucial role in source code
representation. However, due to the large number of nodes in an AST and the
typically deep AST hierarchy, it is challenging to learn the hierarchical
structure of an AST effectively. In this paper, we propose HELoC, a
hierarchical contrastive learning model for source code representation. To
effectively learn the AST hierarchy, we use contrastive learning to allow the
network to predict the AST node level and learn the hierarchical relationships
between nodes in a self-supervised manner, which makes the representation
vectors of nodes with greater differences in AST levels farther apart in the
embedding space. By using such vectors, the structural similarities between
code snippets can be measured more precisely. In the learning process, a novel
GNN (called Residual Self-attention Graph Neural Network, RSGNN) is designed,
which enables HELoC to focus on embedding the local structure of an AST while
capturing its overall structure. HELoC is self-supervised and can be applied to
many source code related downstream tasks such as code classification, code
clone detection, and code clustering after pre-training. Our extensive
experiments demonstrate that HELoC outperforms the state-of-the-art source code
representation models.
