Understanding the Automated Parameter Optimization on Transfer Learning for CPDP: An Empirical Study

02/08/2020
by   Ke Li, et al.
0

Data-driven defect prediction has become increasingly important in software engineering process. Since it is not uncommon that data from a software project is insufficient for training a reliable defect prediction model, transfer learning that borrows data/knowledge from other projects to facilitate the model building at the current project, namely cross-project defect prediction (CPDP), is naturally plausible. Most CPDP techniques involve two major steps, i.e., transfer learning and classification, each of which has at least one parameter to be tuned to achieve their optimal performance. This practice fits well with the purpose of automated parameter optimization. However, there is a lack of thorough understanding about what are the impacts of automated parameter optimization on various CPDP techniques. In this paper, we present the first empirical study that looks into such impacts on 62 CPDP techniques, 13 of which are chosen from the existing CPDP literature while the other 49 ones have not been explored before. We build defect prediction models over 20 real-world software projects that are of different scales and characteristics. Our findings demonstrate that: (1) Automated parameter optimization substantially improves the defect prediction performance of 77% CPDP techniques with a manageable computational cost. Thus more efforts on this aspect are required in future CPDP studies. (2) Transfer learning is of ultimate importance in CPDP. Given a tight computational budget, it is more cost-effective to focus on optimizing the parameter configuration of transfer learning algorithms (3) The research on CPDP is far from mature where it is "not difficult" to find a better alternative by making a combination of existing transfer learning and classification techniques. This finding provides important insights about the future design of CPDP techniques.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/31/2020

BiLO-CPDP: Bi-Level Programming for Automated Model Discovery in Cross-Project Defect Prediction

Cross-Project Defect Prediction (CPDP), which borrows data from similar ...
research
11/08/2022

From fat droplets to floating forests: cross-domain transfer learning using a PatchGAN-based segmentation model

Many scientific domains gather sufficient labels to train machine algori...
research
01/24/2019

Transfer-Learning Oriented Class Imbalance Learning for Cross-Project Defect Prediction

Cross-project defect prediction (CPDP) aims to predict defects of projec...
research
08/19/2023

Evaluating Transfer Learning for Simplifying GitHub READMEs

Software documentation captures detailed knowledge about a software prod...
research
01/31/2018

The Impact of Automated Parameter Optimization on Defect Prediction Models

Defect prediction models---classifiers that identify defect-prone softwa...
research
04/06/2018

Bayesian Hierarchical Modelling for Tailoring Metric Thresholds

Software is highly contextual. While there are cross-cutting `global' le...
research
01/06/2020

Cross-Dataset Design Discussion Mining

Being able to identify software discussions that are primarily about des...

Please sign up or login with your details

Forgot password? Click here to reset