  Software clone detection identifies similar code snippets. It has been an
active research topic that attracts extensive attention over the last two
decades. In recent years, machine learning (ML) based detectors, especially
deep learning-based ones, have demonstrated impressive capability on clone
detection. It seems that this longstanding problem has already been tamed owing
to the advances in ML techniques. In this work, we would like to challenge the
robustness of the recent ML-based clone detectors through code
semantic-preserving transformations. We first utilize fifteen simple code
transformation operators combined with commonly-used heuristics (i.e., Random
Search, Genetic Algorithm, and Markov Chain Monte Carlo) to perform equivalent
program transformation. Furthermore, we propose a deep reinforcement
learning-based sequence generation (DRLSG) strategy to effectively guide the
search process of generating clones that could escape from the detection. We
then evaluate the ML-based detectors with the pairs of original and generated
clones. We realize our method in a framework named CloneGen. CloneGen In
evaluation, we challenge the two state-of-the-art ML-based detectors and four
traditional detectors with the code clones after semantic-preserving
transformations via the aid of CloneGen. Surprisingly, our experiments show
that, despite the notable successes achieved by existing clone detectors, the
ML models inside these detectors still cannot distinguish numerous clones
produced by the code transformations in CloneGen. In addition, adversarial
training of ML-based clone detectors using clones generated by CloneGen can
improve their robustness and accuracy. CloneGen Meanwhile, compared with the
commonly-used heuristics, the DRLSG strategy has shown the best effectiveness
in generating code clones to decrease the detection accuracy of the ML-based
detectors.
