Revizor: A Data-Driven Approach to Automate Frequent Code Changes Based on Graph Matching

08/25/2021
by   Oleg Smirnov, et al.
JetBrains
0

Many code changes that developers make in their projects are repeated and constitute recurrent change patterns. It is of interest to collect such patterns from the version history of open-source repositories and suggest the most useful of them as quick fixes. In this paper, we present Revizor - a tool aimed to build custom plugins for PyCharm, a popular Python IDE. A Revizor-based plugin can take change patterns and highlight potential places for their application in the developer's code editor. If the developer accepts the quick fix, the plugin automatically performs the edit. Our approach uses a graph-based representation of code changes, which allows it to support complex distributed code patterns. Experienced developers have also rated the usability and the performance of such Revizor-based plugin positively. The source code of the tool and test plugin prototype are available on GitHub: https://github.com/JetBrains-Research/revizor. A demonstration video with a short tool description can be found on YouTube: https://youtu.be/5eLs14nco7E.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

12/07/2021

IntelliTC: Automating Type Changes in IntelliJ IDEA

Developers often change the types of program elements. Such a refactorin...
12/21/2020

AC2 – Towards Understanding Architectural Changes in Rapid Releases

Open source projects are adopting faster release cycles that reflect var...
07/06/2020

Sosed: a tool for finding similar software projects

In this paper, we present Sosed, a tool for discovering similar software...
01/20/2022

npm-filter: Automating the mining of dynamic information from npm packages

The static properties of code repositories, e.g., lines of code, depende...
07/18/2021

IDEAL: An Open-Source Identifier Name Appraisal Tool

Developers must comprehend the code they will maintain, meaning that the...
05/04/2021

Interactive Static Software Performance Analysis in the IDE

Detecting performance issues due to suboptimal code during the developme...
02/15/2021

Investigating and Recommending Co-Changed Entities for JavaScript Programs

JavaScript (JS) is one of the most popular programming languages due to ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

In recent decades, Integrated Development Environments (IDEs) have been evolving to provide tools that help developers write and edit code ever more efficiently [19]. Writing code involves making a lot of incremental changes and small fixes. Some of such scrupulous high-concentration tasks can be automated to boost developers’ performance. To this end, IDEs incorporate static analysis tools that find and highlight known problems in code, as well as suggest certain quick fixes as possible solutions to these problems in real time. A lot of works focus on automated program repair (APR) [11], which includes localizing bugs and vulnerabilities in code and fixing them. We aim to show that approaches which are similar to APR can be applied not only to bug fixes, but to almost any code change that is of interest to developers—from stylistic code enhancement to the migration of APIs to a different language version.

A good place to search for possible improvements of code is the history of it being changed. Prior research suggests that code changes are repetitive: from time to time different developers not only face the same kind of problems, but try to solve them in a similar way [12, 13]. Such code changes constitute recurrent change patterns that can be mined from the histories of existing software projects.

Semantic change patterns often differ in structure significantly. While some of them are one-liners or even touch only a single statement [7], others can be quite complex and distributed, i.e., involving isolated tokens from different lines of code or even different scopes connected by data or control dependencies (as in Figure 3). From that point of view, code analysis approaches can benefit greatly from utilizing the fact that code is more than just plain text and work with graph-based code representations that make it possible to keep track and make use of the code’s structure and make decisions about its semantics [13, 1].

In this paper, we investigate whether it might be possible to use graph-based representations of recurrent change patterns for code improvement suggestions within an IDE. For that purpose, we have developed Revizor—a tool that allows to build code enhancement plugins for PyCharm [15], a popular IDE for Python developed by JetBrains. We also propose our prototype plugin built with nine pre-approved code change patterns. For any given change pattern, the plugin uses a fine-grained program dependence graph (fgPDG introduced by Nguyen et al. [13] for mining patterns in Java) of the code fragment before the change to localize its occurrences in the user’s code via building subgraph isomorphisms. If such an occurrence is found, the plugin highlights the code fragment for the user, indicating that a quick fix can be applied. If the user chooses to apply the fix, the plugin can do it by performing a sequence of edit actions, which are generated for each pattern from the versions of code before and after the change.

To evaluate the proposed plugin, we conducted a survey of nine experienced developers. The participants positively rated the plugin’s usability and performance and mostly approved the idea of using recurrent code changes to suggest relevant quick fixes in the IDE.

Ii Background

In this section, we describe the approaches that we adopted to build our pipeline. In Sections II-A and II-B, we characterize the solutions that we considered for the plugin to localize patterns in source code and to apply the respective changes (see the two final stages in Figure 1), and we also contrast the adopted solutions with similar ones. In Section II-C, we relate a prospective graph-based approach to code change patterns mining. Integrating it into a code enhancement pipeline is unprecedented and seems promising.

Ii-a Code Pattern Localization

There exists a number of relevant approaches that aim to mine fix patterns and use them to localize potential code flaws.

Meng et al. presented an approach called LASE [10], where the authors initially built a context-aware edit script from two or more code change examples and then identified appropriate locations for code transformation with a generalized tree-based edit context. To localize such a context in the AST of target code, the authors used the Maximum Common Embedded Subtree Extraction algorithm. A similar approach to localization with a tree-matching algorithm was also used by Bader et al. in their tool called Getafix [2], in which bug fix patterns mined in Java are automated. However, these approaches cannot be used to localize distributed code patterns, which may include many subtrees of an AST.

The authors of DevReplay [18] collected code change patterns via AST comparison, converted the source code of the patterns into regular expressions, and then tried to match any of them with the user’s code to localize possible problems in it. Such method is able to handle multi-line patterns, but still cannot automatically manage control or data flow dependencies between elements of the pattern as graph-based approaches do.

Ii-B Edit Template Application

After a pattern is localized in code and the developer confirms applying the fix, the exposed code fragment is changed in accordance with the appropriate edit script. This is usually done via AST transformations similarly to how it was described by Meng et al. [10, 9]. For that purpose, the authors used edit actions of four types—insert, delete, update and move—generated by a modified version of ChangeDistiller [5], a source code differencing tool for extracting fine-grained edit scripts from two versions of an AST: before and after the change.

Bader et al. [2] used a similar tool called GumTree, and, according to Falleri et al. [4], GumTree represents edit actions in a more accurate and concise way compared to other source code differencing tools.

Nowadays, researchers also widely exploit the ideas of Neural Machine Translation (NMT) that view the task of applying changes as a translation problem from a defective code fragment into a correct one 

[17, 8]

. These approaches are hardly interpretable and require collecting a large number of similar patches to train the model, which is a difficult task. On the contrary, heuristic approaches like the ones that employ AST edit actions need far less input and computational resources to perform.

Ii-C Collection of Recurrent Code Changes Using Graphs

To keep track of semantic features, as well as data and control dependencies between the elements of source code, more complex data structures such as graphs can be used. Nguyen et al. proposed an approach called CPatMiner [13] for mining graph-based change patterns in Java code. The representation of code that they used is called Fine-Grained Program Dependence Graph (fgPDG) and is based on the AST of the source code. Such a graph includes three types of nodes: data nodes (for variables, literals, etc.), operation nodes (for arithmetic expressions, assignments, function calls, etc.), and control nodes (for control statements like if, for, while, etc.). These nodes are linked with additional data and control dependency edges.

To represent code changes, Nguyen et al. introduced the concept of a change graph. A change graph is built using two fgPDGs of the code before and after a given change; corresponding unchanged graph nodes are connected with mapping edges. The authors also suggested a way to use this data structure to build a pattern-mining algorithm. The main idea behind it is to recursively extend each already mined change graph to the most frequently encountered adjacent vertex and then match isomorphic graphs using a hash-based heuristic [14] to put them into one particular pattern.

In our prior work [6], we re-implemented this approach for Python, collected and analyzed fgPDG-based code change patterns from 120 popular GitHub repositories. This allowed us to collect recurrent in-the-wild code changes: code enhancements, bug fixes, refactorings, etc. In this work, our goal was to implement quick-fixing actions so that these graph-based changes could be applied automatically in the developer’s code in the IDE.

Fig. 1: An overview of the proposed approach.

Iii Implementation

Iii-a Pipeline Overview

We have implemented an approach to enhancing IDEs with valuable up-to-date code improvement suggestions in a data-driven way. Our contribution is Revizor, a tool that allows to create custom plugins with pre-approved quick fixes for Python code. We also built a prototype plugin for PyCharm, a popular Python IDE based on the IntelliJ Platform.111https://plugins.jetbrains.com/docs/intellij/intellij-platform.html The plugin is created using code change patterns and respective code samples mined with PythonChangeMiner [6], which analyzes the graph-based representation of code changes in Git repositories and detects change patterns without any prior specification of what is worth changing in Python code, thus sparing the necessity to devise and manually write code enhancement rules. The proposed approach allows plugins to locate distributed code patterns, i.e., the patterns involving isolated tokens that are located on multiple lines of code and connected by some data or control dependencies. Using fgPDGs enables greater matching flexibility and structural awareness compared to general regex patterns.

The full pipeline behind Revizor is shown in Figure 1. The steps to build the plugin are as follows:

  1. Collecting: Collect graph-based patterns of code changes in Python and choose the ones that should be automated.

  2. Preprocessing: Build the Revizor plugin using any type of sources from step 1. If in future any new change patterns are added, re-build the plugin.

The steps are described in more detail in the next subsections. For more information on how to build a plugin with Revizor, see its README on GitHub [16].

After the plugin is installed in the IDE, it tracks developer’s actions when a .py file is opened or changed in the code editor. Namely, the plugin:

  1. Builds an fgPDG for each function in the developer’s code using its PSI tree (an enriched form of a concrete syntax tree used in the IntelliJ Platform).222https://plugins.jetbrains.com/docs/intellij/psi.html It is performed on-the-fly using IntelliJ’s mechanism called code inspections.333https://plugins.jetbrains.com/docs/intellij/code-inspections.html

  2. Checks such graphs for possible subgraph isomorphisms with the before version of each available change pattern stored in the plugin’s resources.

  3. Highlights the corresponding code tokens in the editor if such an isomorphism is detected and suggests the respective improvement.

  4. Performs the improvements confirmed by the developer using a sequence of edit actions extracted with GumTree during the preprocessing step.

Iii-B Change Pattern Collecting

In our prior work [6], we had gathered a dataset of 120 popular GitHub repositories based on their domain, length of the commit history, age of the project, and number of contributors. Finally, we had discovered a total of 7,481 code change patterns. To understand their semantics better, we manually evaluated and categorized 803 patterns that appeared in at least two projects. From this pool, we selected nine patterns presented in Table 2 for the evaluation of our test plugin’s prototype according to the following criteria: the patterns had different structure and semantics and constituted good examples of what software engineers we consulted thought worth automating. The selected changes are related to developers’ best practices, which evolve rapidly with any language and are rarely documented promptly. Also, fixes for such changes as Enumerate are not easy to implement because the pattern could be distributed. Revizor-based plugins not only detect such unique change patterns but fix them as shown in Figure 3. Finding change patterns can be automated to a large extent and it is a promising direction of future work.

Fig. 2: The examples of the chosen nine change patterns for Revizor-based plugin prototype evaluation.

Iii-C Pattern preprocessing

Before the mined change patterns can be used in the plugin, they should be preprocessed with our tool. The process is fully automated except for the specification of tooltip annotations for each pattern that will appear in the IDE.

Iii-C1 Assignment of matching modes

During preprocessing of the supplied graphs, Revizor automatically specifies how data vertices from the pattern should be matched with the ones from developer’s code when a Revizor-based plugin looks for “familiar” patterns in code. Such rules are called matching modes and take into account the vertices’ labels, positions, and neighbours in the fgPDG.

Examples. Some user-defined variable names that refer to the same data element (e.g., a list may be named lst, data or items) do not need to be exactly matched during subgraph isomorphism search, and therefore we assigned such vertices with the matchanylabel matching mode. Other variable names should be considered as having a partial match with a common suffix, e.g., dict.keys and vocab.keys (the matchlongestcommonsuffix mode). Some should always be matched exactly as they are built-in or external library Python functions and attributes, e.g., collections.Callable or np.log (the matchoriginallabels mode).

Iii-C2 Generating edit actions

We use GumTree [4] to extract sequences of edit actions from PSI trees of the before and after code fragments related to the change (they are stored by the miner together with the respective graphs). Edit actions keep references to the corresponding vertices of the PSI tree before the change, making it possible to apply these actions to code merely one by one.

While GumTree extracts all edits from such code changes (most of them potentially unrelated to our pattern), we need to get only necessary actions. We do so by calculating a Longest Common Edit Operation Subsequence with generalized identifiers by iteratively comparing the extracted edit actions sequences pairwise. A similar process was described in detail in the work about LASE [10], but instead of keeping an edit context, we use the isomorphic mappings between fgPDGs for all encountered fragments of the pattern.

Fig. 3: An example of applying a graph-based change pattern in our plugin: replacing the range function call in the for loop condition with enumerate. The highlighted code tokens are placed in different lines of code and include all the vertices of the detected fgPDG with respect to the data flow dependencies, e.g., data declaration in line 37.

Iii-C3 Extending fgPDGs

When an appropriate subsequence of edit actions is extracted and saved, we extend the originally mined fgPDG of the pattern with additional vertices that appeared in these edits. This is done because some of them may contain PSI nodes that did not even exist in the original fgPDG of the pattern, such as parents of the moved or inserted vertices in the PSI tree.

Finally, the preprocessing script automatically saves the assigned matching modes and edit actions, as well as the extended graph and the manually provided description of each pattern. After loading all of them as resources, the plugin is ready to go.

Iii-D Possible Application

A Revizor-based plugin may meet the following purposes of a code standardization linter:

  • Self-education (individual level): Use our prototype with a selection of promising code changes from popular GitHub repositories.

  • Enforcement of style guidelines (team/company level): Build a plugin around mined or manually created patterns of interest.

  • Introduction of fresh high-quality inspirations from other developers (individual/team/company/education level): Build a plugin around mined and sifted patterns from relevant code repositories.

Iv Evaluation

As a preliminary study, we asked nine developers to install our plugin in their PyCharm IDE and test it on an example project [3], which contained manually selected code snippets from several Python projects where the chosen patterns were encountered during mining. The participants were requested to find and perform all the suggested code changes, considering the usability of the tool. All the developers had from two to five years of professional experience and confirmed that they often used intention actions in the IntelliJ-based IDEs to improve their code quality. The survey participants also agreed that the idea of using the most common code changes mined from GitHub as quick fixes in the IDE looked potentially useful.

We asked them to rate different aspects of the plugin’s performance with one of the four responses: very dissatisfied (1 point), not really satisfied (2 points), rather satisfied (3 points) and very satisfied (4 points). The average score we received regarding the correctness of the plugin’s edit operations was 3.66 out of 4, and the overall usability was rated at 3.77 out of 4. Developers also highly evaluated the performance of Revizor in terms of not affecting the overall IDE performance (3.88 out of 4). The lowest rated feature of the plugin was the pattern’s visualization part (3.33 out of 4), as it turned out that the current approach to highlighting complex distributed patterns sometimes could be confusing for the developers.

V Conclusion and Future Work

In this paper, we presented an extendable approach to automated code enhancement. We proposed a data-driven tool called Revizor for building static code analysis plugins that use subgraph isomorphism mappings for pattern localization and GumTree edit actions to automate changes. The tool uses frequent Python change patterns mined from GitHub repositories. We also created a test prototype of the plugin for a popular Python IDE called PyCharm and evaluated it on several experienced developers who approved its usability.

We received a lot of feedback about improving the UI/UX of the plugin, i.e., how it treats the patterns, including an idea to highlight distributed patterns in a more intelligent way: first highlight the key token and when the user clicks it, highlight the other parts of the pattern. Also, we received feature requests such as to apply the selected change across the whole project or to suppress the suggestions for particular scopes of code. We aim to implement these features in future.

Overall, the described data-driven approach is potentially extendable to any other programming languages, but in order to capture any data or control dependencies in the code, the extended approach should be tightly dependent on the language grammar. We also plan to support the unified fgPDG representations in our tool for other languages using the PSI.

References

  • [1] M. Allamanis, M. Brockschmidt, and M. Khademi (2017) Learning to represent programs with graphs. arXiv preprint arXiv:1711.00740. Cited by: §I.
  • [2] J. Bader, A. Scott, M. Pradel, and S. Chandra (2019) Getafix: Learning to fix bugs automatically. Proceedings of the ACM on Programming Languages 3 (OOPSLA), pp. 1–27. Cited by: §II-A, §II-B.
  • [3] (2021)(Website) External Links: Link Cited by: §IV.
  • [4] J. Falleri, F. Morandat, X. Blanc, M. Martinez, and M. Monperrus (2014) Fine-grained and accurate source code differencing. In Proceedings of the 29th ACM/IEEE international conference on Automated software engineering, pp. 313–324. Cited by: §II-B, §III-C2.
  • [5] B. Fluri, M. Wursch, M. PInzger, and H. Gall (2007) Change distilling: tree differencing for fine-grained source code change extraction. IEEE Transactions on software engineering 33 (11), pp. 725–743. Cited by: §II-B.
  • [6] Y. Golubev, J. Li, V. Bushev, T. Bryksin, and I. Ahmed (2021) Changes from the trenches: should we automate them?. arXiv preprint arXiv:2105.10157. Cited by: §II-C, §III-A, §III-B.
  • [7] R. Karampatsis and C. Sutton (2020) How often do single-statement bugs occur? The ManySStuBs4J dataset. In Proceedings of the 17th International Conference on Mining Software Repositories, pp. 573–577. Cited by: §I.
  • [8] T. Lutellier, L. Pang, V. H. Pham, M. Wei, and L. Tan (2019) ENCORE: ensemble learning using convolution neural machine translation for automatic program repair. arXiv preprint arXiv:1906.08691. Cited by: §II-B.
  • [9] N. Meng, M. Kim, and K. S. McKinley (2011) Systematic editing: generating program transformations from an example. ACM SIGPLAN Notices 46 (6), pp. 329–342. Cited by: §II-B.
  • [10] N. Meng, M. Kim, and K. S. McKinley (2013) LASE: locating and applying systematic edits by learning from examples. In 2013 35th International Conference on Software Engineering (ICSE), pp. 502–511. Cited by: §II-A, §II-B, §III-C2.
  • [11] M. Monperrus (2020) The living review on automated program repair. Cited by: §I.
  • [12] H. A. Nguyen, A. T. Nguyen, T. T. Nguyen, T. N. Nguyen, and H. Rajan (2013) A study of repetitiveness of code changes in software evolution. In 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 180–190. Cited by: §I.
  • [13] H. A. Nguyen, T. N. Nguyen, D. Dig, S. Nguyen, H. Tran, and M. Hilton (2019) Graph-based mining of in-the-wild, fine-grained, semantic code change patterns. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), pp. 819–830. Cited by: §I, §I, §I, §II-C.
  • [14] H. A. Nguyen, T. T. Nguyen, N. H. Pham, J. M. Al-Kofahi, and T. N. Nguyen (2009)

    Accurate and efficient structural characteristic feature extraction for clone detection

    .
    In International Conference on Fundamental Approaches to Software Engineering, pp. 440–455. Cited by: §II-C.
  • [15] (2021)(Website) External Links: Link Cited by: §I.
  • [16] (2021)(Website) External Links: Link Cited by: §III-A.
  • [17] M. Tufano, C. Watson, G. Bavota, M. D. Penta, M. White, and D. Poshyvanyk (2019) An empirical study on learning bug-fixing patches in the wild via neural machine translation. ACM Transactions on Software Engineering and Methodology (TOSEM) 28 (4), pp. 1–29. Cited by: §II-B.
  • [18] Y. Ueda, T. Ishio, A. Ihara, and K. Matsumoto (2020) DevReplay: automatic repair with editable fix pattern. arXiv preprint arXiv:2005.11040. Cited by: §II-A.
  • [19] I. Zayour and H. Hajjdiab (2013) How much integrated development environments (IDEs) improve productivity?. JSW 8 (10), pp. 2425–2431. Cited by: §I.