Challenging Machine Learning-based Clone Detectors via Semantic-preserving Code Transformations

by   Weiwei Zhang, et al.

Software clone detection identifies similar code snippets. It has been an active research topic that attracts extensive attention over the last two decades. In recent years, machine learning (ML) based detectors, especially deep learning-based ones, have demonstrated impressive capability on clone detection. It seems that this longstanding problem has already been tamed owing to the advances in ML techniques. In this work, we would like to challenge the robustness of the recent ML-based clone detectors through code semantic-preserving transformations. We first utilize fifteen simple code transformation operators combined with commonly-used heuristics (i.e., Random Search, Genetic Algorithm, and Markov Chain Monte Carlo) to perform equivalent program transformation. Furthermore, we propose a deep reinforcement learning-based sequence generation (DRLSG) strategy to effectively guide the search process of generating clones that could escape from the detection. We then evaluate the ML-based detectors with the pairs of original and generated clones. We realize our method in a framework named CloneGen. CloneGen In evaluation, we challenge the two state-of-the-art ML-based detectors and four traditional detectors with the code clones after semantic-preserving transformations via the aid of CloneGen. Surprisingly, our experiments show that, despite the notable successes achieved by existing clone detectors, the ML models inside these detectors still cannot distinguish numerous clones produced by the code transformations in CloneGen. In addition, adversarial training of ML-based clone detectors using clones generated by CloneGen can improve their robustness and accuracy. CloneGen Meanwhile, compared with the commonly-used heuristics, the DRLSG strategy has shown the best effectiveness in generating code clones to decrease the detection accuracy of the ML-based detectors.



There are no comments yet.


page 1

page 2

page 3

page 4


Towards Making Deep Learning-based Vulnerability Detectors Robust

Automatically detecting software vulnerabilities in source code is an im...

Towards Automatic Learning of Heuristics for Mechanical Transformations of Procedural Code

The current trends in next-generation exascale systems go towards integr...

ML-based IoT Malware Detection Under Adversarial Settings: A Systematic Evaluation

The rapid growth of the Internet of Things (IoT) devices is paralleled b...

An ensemble learning approach for software semantic clone detection

Code clone is a serious problem in software and has the potential to sof...

SearchFromFree: Adversarial Measurements for Machine Learning-based Energy Theft Detection

Energy theft causes large economic losses to utility companies around th...

Provable Robust Learning Based on Transformation-Specific Smoothing

As machine learning systems become pervasive, safeguarding their securit...

Upper Bounding GED via Transformations to LSAPE Based on Rings and Machine Learning

The graph edit distance (GED) is a flexible distance measure which is wi...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.