Deep Transfer Learning for Source Code Modeling

10/12/2019
by   Yasir Hussain, et al.
21

In recent years, deep learning models have shown great potential in source code modeling and analysis. Generally, deep learning-based approaches are problem-specific and data-hungry. A challenging issue of these approaches is that they require training from starch for a different related problem. In this work, we propose a transfer learning-based approach that significantly improves the performance of deep learning-based source code models. In contrast to traditional learning paradigms, transfer learning can transfer the knowledge learned in solving one problem into another related problem. First, we present two recurrent neural network-based models RNN and GRU for the purpose of transfer learning in the domain of source code modeling. Next, via transfer learning, these pre-trained (RNN and GRU) models are used as feature extractors. Then, these extracted features are combined into attention learner for different downstream tasks. The attention learner leverages from the learned knowledge of pre-trained models and fine-tunes them for a specific downstream task. We evaluate the performance of the proposed approach with extensive experiments with the source code suggestion task. The results indicate that the proposed approach outperforms the state-of-the-art models in terms of accuracy, precision, recall, and F-measure without training the models from scratch.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 2

page 7

page 10

page 13

page 15

page 16

page 17

page 21

12/04/2021

Bridging Pre-trained Models and Downstream Tasks for Source Code Understanding

With the great success of pre-trained models, the pretrain-then-finetune...
04/05/2019

On the Feasibility of Transfer-learning Code Smells using Deep Learning

Context: A substantial amount of work has been done to detect smells in ...
12/30/2021

Deep Transfer-Learning for patient specific model re-calibration: Application to sEMG-Classification

Accurate decoding of surface electromyography (sEMG) is pivotal for musc...
05/23/2019

An Investigation of Transfer Learning-Based Sentiment Analysis in Japanese

Text classification approaches have usually required task-specific model...
11/18/2019

patch2vec: Distributed Representation of Code Changes

Deep learning methods, which have found successful applications in field...
05/16/2021

SLGPT: Using Transfer Learning to Directly Generate Simulink Model Files and Find Bugs in the Simulink Toolchain

Finding bugs in a commercial cyber-physical system (CPS) development too...
05/27/2020

Automatic salt deposits segmentation: A deep learning approach

One of the most important applications of seismic reflection is the hydr...

Code Repositories

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.