Unsupervised Learning of General-Purpose Embeddings for Code Changes

06/03/2021
by   Mikhail Pravilov, et al.
0

Applying machine learning to tasks that operate with code changes requires their numerical representation. In this work, we propose an approach for obtaining such representations during pre-training and evaluate them on two different downstream tasks - applying changes to code and commit message generation. During pre-training, the model learns to apply the given code change in a correct way. This task requires only code changes themselves, which makes it unsupervised. In the task of applying code changes, our model outperforms baseline models by 5.9 percentage points in accuracy. As for the commit message generation, our model demonstrated the same results as supervised models trained for this specific task, which indicates that it can encode code changes well and can be improved in the future by pre-training on a larger dataset of easily gathered code changes.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/08/2023

CCRep: Learning Code Change Representations via Pre-Trained Code Model and Query Back

Representing code changes as numeric feature vectors, i.e., code change ...
research
03/17/2022

CodeReviewer: Pre-Training for Automating Code Review Activities

Code review is an essential part to software development lifecycle since...
research
10/07/2022

Supervised and Unsupervised Learning of Audio Representations for Music Understanding

In this work, we provide a broad comparative analysis of strategies for ...
research
05/24/2021

One4all User Representation for Recommender Systems in E-commerce

General-purpose representation learning through large-scale pre-training...
research
08/11/2021

Learning Oculomotor Behaviors from Scanpath

Identifying oculomotor behaviors relevant for eye-tracking applications ...
research
04/27/2023

DataComp: In search of the next generation of multimodal datasets

Large multimodal datasets have been instrumental in recent breakthroughs...
research
12/02/2021

GraphCode2Vec: Generic Code Embedding via Lexical and Program Dependence Analyses

Code embedding is a keystone in the application of machine learning on s...

Please sign up or login with your details

Forgot password? Click here to reset