Automatic Pull Request Title Generation

06/21/2022
by   Ting Zhang, et al.
0

Pull Requests (PRs) are a mechanism on modern collaborative coding platforms, such as GitHub. PRs allow developers to tell others that their code changes are available for merging into another branch in a repository. A PR needs to be reviewed and approved by the core team of the repository before the changes are merged into the branch. Usually, reviewers need to identify a PR that is in line with their interests before providing a review. By default, PRs are arranged in a list view that shows the titles of PRs. Therefore, it is desirable to have a precise and concise title, which is beneficial for both reviewers and other developers. However, it is often the case that developers do not provide good titles; we find that many existing PR titles are either inappropriate in length (i.e., too short or too long) or fail to convey useful information, which may result in PR being ignored or rejected. Therefore, there is a need for automatic techniques to help developers draft high-quality titles. In this paper, we introduce the task of automatic generation of PR titles. We formulate the task as a one-sentence summarization task. To facilitate the research on this task, we construct a dataset that consists of 43,816 PRs from 495 GitHub repositories. We evaluated the state-of-the-art summarization approaches for the automatic PR title generation task. We leverage ROUGE metrics to automatically evaluate the summarization approaches and conduct a manual evaluation. The experimental results indicate that BART is the best technique for generating satisfactory PR titles with ROUGE-1, ROUGE-2, and ROUGE-L F1-scores of 47.22, 25.27, and 43.12, respectively. The manual evaluation also shows that the titles generated by BART are preferred.

READ FULL TEXT

page 1

page 9

research
10/25/2021

Generating GitHub Repository Descriptions: A Comparison of Manual and Automated Approaches

Given the vast number of repositories hosted on GitHub, project discover...
research
06/23/2022

AutoPRTitle: A Tool for Automatic Pull Request Title Generation

With the rise of the pull request mechanism in software development, the...
research
08/18/2021

More Than React: Investigating The Role of EmojiReaction in GitHub Pull Requests

Context: Open source software development has become more social and col...
research
01/17/2021

GENIE: A Leaderboard for Human-in-the-Loop Evaluation of Text Generation

Leaderboards have eased model development for many NLP datasets by stand...
research
08/02/2022

Together or Apart? Investigating a mediator bot to aggregate bot's comments on pull requests

Software bots connect users and tools, streamlining the pull request rev...
research
02/26/2021

Neural Code Summarization

Code summarization is the task of generating readable summaries that are...
research
08/11/2018

ESSMArT Way to Manage User Requests

Quality and market acceptance of software products is strongly influence...

Please sign up or login with your details

Forgot password? Click here to reset