  With the advancement of deep learning (DL) in various fields, there are many
attempts to reveal software vulnerabilities by data-driven approach.
Nonetheless, such existing works lack the effective representation that can
retain the non-sequential semantic characteristics and contextual relationship
of source code attributes. Hence, in this work, we propose XGV-BERT, a
framework that combines the pre-trained CodeBERT model and Graph Neural Network
(GCN) to detect software vulnerabilities. By jointly training the CodeBERT and
GCN modules within XGV-BERT, the proposed model leverages the advantages of
large-scale pre-training, harnessing vast raw data, and transfer learning by
learning representations for training data through graph convolution. The
research results demonstrate that the XGV-BERT method significantly improves
vulnerability detection accuracy compared to two existing methods such as
VulDeePecker and SySeVR. For the VulDeePecker dataset, XGV-BERT achieves an
impressive F1-score of 97.5%, significantly outperforming VulDeePecker, which
achieved an F1-score of 78.3%. Again, with the SySeVR dataset, XGV-BERT
achieves an F1-score of 95.5%, surpassing the results of SySeVR with an
F1-score of 83.5%.
