  With the increasing release of powerful language models trained on large code
corpus (e.g. CodeBERT was trained on 6.4 million programs), a new family of
mutation testing tools has arisen with the promise to generate more "natural"
mutants in the sense that the mutated code aims at following the implicit rules
and coding conventions typically produced by programmers. In this paper, we
study to what extent the mutants produced by language models can semantically
mimic the observable behavior of security-related vulnerabilities (a.k.a.
Vulnerability-mimicking Mutants), so that designing test cases that are failed
by these mutants will help in tackling mimicked vulnerabilities. Since
analyzing and running mutants is computationally expensive, it is important to
prioritize those mutants that are more likely to be vulnerability mimicking
prior to any analysis or test execution. Taking this into account, we introduce
VMMS, a machine learning based approach that automatically extracts the
features from mutants and predicts the ones that mimic vulnerabilities. We
conducted our experiments on a dataset of 45 vulnerabilities and found that
16.6% of the mutants fail one or more tests that are failed by 88.9% of the
respective vulnerabilities. More precisely, 3.9% of the mutants from the entire
mutant set are vulnerability-mimicking mutants that mimic 55.6% of the
vulnerabilities. Despite the scarcity, VMMS predicts vulnerability-mimicking
mutants with 0.63 MCC, 0.80 Precision, and 0.51 Recall, demonstrating that the
features of vulnerability-mimicking mutants can be automatically learned by
machine learning models to statically predict these without the need of
investing effort in defining such features.
