Unit Test Case Generation with Transformers

09/11/2020
by   Michele Tufano, et al.
0

Automated Unit Test Case generation has been the focus of extensive literature within the research community. Existing approaches are usually guided by the test coverage criteria, generating synthetic test cases that are often difficult to read or understand for developers. In this paper we propose AthenaTest, an approach that aims at generating unit test cases by learning from real-world, developer-written test cases. Our approach relies on a state-of-the-art sequence-to-sequence transformer model which is able to write useful test cases for a given method under test (i.e., focal method). We also introduce methods2test - the largest publicly available supervised parallel corpus of unit test case methods and corresponding focal methods in Java, which comprises 630k test cases mined from 70k open-source repositories hosted on GitHub. We use this dataset to train a transformer model to translate focal methods into the corresponding test cases. We evaluate the ability of our model in generating test cases using natural language processing as well as code-specific criteria. First, we assess the quality of the translation compared to the target test case, then we analyze properties of the test case such as syntactic correctness and number and variety of testing APIs (e.g., asserts). We execute the test cases, collect test coverage information, and compare them with test cases generated by EvoSuite and GPT-3. Finally, we survey professional developers on their preference in terms of readability, understandability, and testing effectiveness of the generated test cases.

READ FULL TEXT

page 1

page 9

research
09/11/2020

Generating Accurate Assert Statements for Unit Test Cases using Pretrained Transformers

Unit testing represents the foundational basis of the software testing p...
research
03/23/2022

Methods2Test: A dataset of focal methods mapped to test cases

Unit testing is an essential part of the software development process, w...
research
11/03/2014

NESTA, The NICTA Energy System Test Case Archive

In recent years the power systems research community has seen an explosi...
research
02/20/2023

A3Test: Assertion-Augmented Automated Test Case Generation

Test case generation is an important activity, yet a time-consuming and ...
research
09/21/2022

PTSG: a test generation tool based on extended finite state machine

The Extended Finite State Machine (EFSM) is one of the most popular mode...
research
08/31/2023

Effective Test Generation Using Pre-trained Large Language Models and Mutation Testing

One of the critical phases in software development is software testing. ...
research
09/14/2023

Coverage Goal Selector for Combining Multiple Criteria in Search-Based Unit Test Generation

Unit testing is critical to the software development process, ensuring t...

Please sign up or login with your details

Forgot password? Click here to reset