Automated Regression Unit Test Generation for Program Merges

by   Tao Ji, et al.

Merging other branches into the current working branch is common in collaborative software development. However, developers still heavily rely on the textual merge tools to handle the complicated merge tasks. The latent semantic merge conflicts may fail to be detected and degrade the software quality. Regression testing is able to prevent regression faults and has been widely used in real-world software development. However, the merged software may fail to be well examined by rerunning the existing whole test suite. Intuitively, if the test suite fails to cover the changes of different branches at the same time, the merge conflicts would fail to be detected. Recently, it has been proposed to conduct verification on 3-way merges, but this approach does not support even some common cases such as different changes made to different parts of the program. In this paper, we propose an approach of regression unit test generation specifically for checking program merges according to our proposed test oracles. And our general test oracles support us to examine not only 3-way merges, but also 2-way and octopus merges. Considering the conflicts may arise in other locations besides changed methods of the project, we design an algorithm to select UUTs based on the dependency analysis of the whole project. On this basis, we implement a tool called TOM to generate unit tests for Java program merges. We also design the benchmark MCon4J consisting of 389 conflict 3-way merges and 389 conflict octopus merges to facilitate further studies on this topic. The experimental results show that TOM finds 45 conflict 3- way merges and 87 conflicts octopus merges, while the verification based tool fails to work on MCon4J.



There are no comments yet.


page 1

page 2

page 3

page 4


Evolutionary Conflict Checking

During the software evolution, existing features may be adversely affect...

Verifying Semantic Conflict-Freedom in Three-Way Program Merges

Even though many programmers rely on 3-way merge tools to integrate chan...

BDCI: Behavioral Driven Conflict Identification

Source Code Management (SCM) systems support software evolution by provi...

Predicting Merge Conflicts in Collaborative Software Development

Background. During collaborative software development, developers often ...

Can Pre-trained Language Models be Used to Resolve Textual and Semantic Merge Conflicts?

Program merging is standard practice when developers integrate their ind...

Can Program Synthesis be Used to Learn Merge Conflict Resolutions? An Empirical Analysis

Forking structure is widespread in the open-source repositories and that...

Extracting Formal Specifications to Strenghten Type Behaviour Testing

Testing has become an indispensable activity of software development, ye...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

Developers utilize the version control systems to make their own changes and accept contributions from other developers. During the process, conflicts may arise once developers merge other branches into the current working branch. Conflicts annoy the developers and developers have to take much efforts and be careful to deal with these conflicts (McKee et al., 2017). After decades of hard working on detecting merge conflicts, various tools have been proposed (Mens, 2002; Brun et al., 2011; Guimarães and Silva, 2012; Kasi and Sarma, 2013) to assist developers in detecting conflicts, reviewing changes and resolving conflicts. However, considering the generality and usability of these tools, developers still rely on textual merge tools (e.g., those integrated in version control systems) to deal with their daily merge work (McKee et al., 2017). In addition, a recent study conducted by Ahmed et al. (Ahmed et al., 2017) shows that merges contain more code smells once conflicts arise. This reality motivates us to figure out some general and effective method that has the potential to be widely used in real-world development.

Regression testing has been widely used to prevent regression faults and ensure the software quality after changes are made to the software. Obviously, regression testing also can be used to detect conflicts after merging branches. Once one test fails on the merge version but passes on all of the parent versions, this test reveals the merge conflicts. Along this direction, Brun et al. (Brun et al., 2011)classify merge conflicts into textual and higher-order conflicts (i.e., build and test conflicts), and tell whether test conflicts arise according to the results of rerunning the existing test suite. As is well known, maintaining a high-quality test suite requires much efforts. Hence, automatically generating unit test cases has been studied extensively. And, to evolve the test suite with the software, regression unit test generation is proposed to ensure that the software does not have other unexpected behaviors brought by new changes.

It seems that ideally rerunning the test suite is enough for detecting merge conflicts if we have a pretty high-quality test suite before merging branches. However, in practice, the test suite may still have a high probability to miss the merge conflicts due to the work flow of the collaborative development. Imagine that two branches are developed by different developers, the newly added or changed test cases may not well cover the changes made by the other branch since the developers are not aware of others’ work. In other words, if there exists one execution path that includes both changes of two branches, this path would not be covered by the existing test suite. As a result, the conflicts between these changes fail to be detected by running the existing test suite. If developers carefully examine the relationships between changes of different branches with the help of change impact analysis tools, the conflicts can be reduced. However, a recent survey 

(Jiang et al., 2017) shows that developers do not use any change impact analysis tool in their daily debugging work, although they think these tools are much helpful.

Recently, Sousa et al. (Sousa et al., 2018) have made progress on guaranteeing the quality of 3-way merges, by proposing the contract of semantic conflict freedom and developing the tool SafeMerge to verify whether one 3-way merge meets this contract. However, the state-of-the-art verification approach still has a set of limitations and challenges in verifying program merges. First, besides 3-way merges, Git also supports to merge two branches without the common ancestor (i.e., 2-way merge) or more than 2 branches (i.e., octopus merge).111 SafeMerge only supports 3-way merge ant thus fails to deal with other common merge scenarios. Second, SafeMerge only works on those cases that two branches make changes to the same Java method. Intuitively, changes made to different methods also may bring conflicts if these two methods are invoked along the same execution path. Third, as described in (Sousa et al., 2018), SafeMerge has a set of limitations on changes made to method signatures, the analysis scope and exceptions. In addition, considering the challenges for static analysis of Java reflection, verification on Java program merges involving Java dynamic features is not sound (Landman et al., 2017).

In this paper, we propose general test oracles for merges inspired from the contract of semantic conflict freedom and make the oracles applicable for all real-world merge scenarios (i.e., 2-way, 3-way and octopus merges). After that, based on our proposed test oracles, we address the problems that how to find the UUTs (Unit Under Testing) from the whole project and which variant involved in the merge scenario should be used to generate test cases. We implement a tool named TOM (Testing on Merges) to automatically find the impacted methods due to changes and then generate test cases to reveal conflicts. Moreover, we construct a benchmark named MCon4J (Merge Conflicts for Java). MCon4J contains a total of 389 three-way merges and 389 octopus merges respectively, in each of which merge conflicts exist. Then, we conduct experiments to examine the effectiveness of TOM.

In summary, our contributions are as follows:

  • We propose to apply the regression unit test generation for detecting merge conflicts as the supplement of existing regression testing that does not consider merges specifically;

  • We propose the notion of general test oracles for checking semantic conflicts in merges, which supports 2-way, 3-way and octopus merges;

  • We design an algorithm to efficiently find the UUTs from the whole project based on dependency analysis such that we can concentrate on generating test cases to detect semantic conflicts over these UUTs;

  • We design the benchmark MCon4j consisting of 389 conflict 3-way merges and 389 conflict octopus merges, to facilitate further studies on detecting semantic conflicts;

  • Experimental results show that our tool TOM finds 45 conflict 3-way merges and 87 octopus merges.

The rest of the paper is structured as follows. First we give our motivation (Section II). Then we detail our proposed approach (Section III) and evaluation (Section IV). After that, we present the discussion (Section V) and related work (Section VI). Finally, we conclude (Section VII).

2. Motivation

In this section, we discuss the shortcomings of existing methods that can be used to detect merge conflicts, which motivates and inspires our work.

2.1. Verification on Three-way Merges

First, we recall the notion of semantic conflict freedom, proposed recently by Sousa et al. (Sousa et al., 2018) for verifying 3-way merges.

Definition 2.1 (Semantic Conflict Freedom) Suppose that we are given four program versions representing the base program, its two variants, and the merge respectively. We say that is semantically conflict-free, if for all valuations such that:

the following conditions hold for all (where represents the outputs):

(1) If [(,)][(,)], then [(,)]=[(,)]

(2) If [(,)][(,)], then [(,)]=[(,)]

(3) Otherwise, [(,)] = [(,)] = [(,)] = [(,)]

Specifically, if one variable’s value returned by variant A (resp. B) differs from the same variable’s value returned by the base O, then this variable’s return value of the merge M should agree with A (resp. B).

Figure 1. The merge has parents merged and these parents may have the common ancestor.

Real-world Merge Scenarios. In real-world software development, Git is the most popular distributed version control system. Different from the 3-way merge, 2-way merge has two branches merged without the common ancestor. Git supports merging the other unrelated branch222 into the working branch, which is one typical case of the 2-way merge. Imagine that two branches both add methods with the same name compared to their ancestor. Without any common ancestors, we also consider this merge as 2-way merge. Moreover, Git supports merging multi-branches into the working branch, which is called the octopus merge. However, we cannot use the notion of semantic conflict freedom in Definition 2.1 for verifying 2-way and octopus merges. The meaning of n-way merge has some ambiguity, for example, Leßenich et al (Leßenich et al., 2017) denote the octopus merge as n-way merge. Hence, in our paper, we use the n-parent merge to describe the real-world merge scenarios, as shown in Fig. 1.

False Positives. In real-world merge scenarios, we cannot get the merge version successfully when two variants conflicts. To have the final merge, developers may make a concession or try to figure out other ways. However, the semantic conflict freedom is defined based on the idea that the merge should reserve all the changes of different branches. Since other determination on the merge’s behavior has been introduced when developers resolve conflicts, the notion of semantic conflict freedom does not fit.

According to the definition of semantic conflict freedom, once two variants change the same variable’s value to different values, we can say the semantic conflict arises without analyzing the merge result. Consider one example calculating one person’s total income. The base version gets the income by “income = salary”, the first variant changes it to “income = salary + stock” and the second changes it to “income = salary + rent”. Actually, two assignments do not conflict with each other considered developers’ intention, and we shall use “income = salary + stock + rent” as the merge result. In this case, this merge is not semantic conflict free while it may be the best merge candidate. Moreover, the verification tool SafeMerge333Its latest commit is b2bac46ada till the date that we submit this paper, and we conduct experiments on this version. reports conflicts without providing any counterexamples, which will increase developers’ burden on checking the merge result. Hence, we consider proposing the test generation approach to generate tests that cover the conflict parts. Then, it will be much easier for developers to examine the merges.

Figure 2. Semantic conflicts arise in this merge.

False Negatives. SafeMerge works at the method level and only works when both branches make changes to the same method. Hence, we wonder whether semantic conflicts arise when two branches modify the different methods. As shown in the Fig. 2, this Java class has two setter methods. The first variant changes the body of setX to “this.x=x+1;”, while the second variant changes the body of setY to “this.y=y+1;”. Obviously, we can merge them successfully by those textual merge tools, while SafeMerge will not report the semantic conflict for the method getSum. However, given the same input for four versions of getSum, the returned results actually violate the semantic conflict freedom defined in Definition 2.1. Imagine some more complicated cases that one method invokes changed methods indirectly. If we want to improve SafeMerge to reduce the false negatives of above cases, we should analyze the invoked changed methods, which would increase the complexity of verification and may bring false positives due to the limitation of current verification techniques.

2.2. Regression Testing

Regression testing is used to make sure that the changes made to software are intended and do not introduce any unexpected behavior. In the ideal case, developers of different branches add new tests to represent their intention on changes in both two branches, and after merging these two branches, developers can simply rerun the whole test suite to make sure the software works well without any failure. However, in practice, since developers may be not aware of the other changes introduced by different branches in advance, the added test cases may not cover those changes. As a result, rerunning the test suite may not expose the conflict introduced by the merged branch.

Existing works on automated regression unit test generation focus on the different parts between two versions. And, to the best of our knowledge, there is no prior work aiming to automatically generate regression unit tests for the merge scenarios. Obviously, it is relatively easy for developers to investigate the program behavior by examining test cases. We wonder whether we can make use of the recent advance of verification on 3-way merges and regression testing to generate test cases to reveal conflicts for n-parent merges.

3. Proposed Approach

Similarly, as for the general problem of test generation, we need to figure out what to test and where to test. In this section, we first propose our notion of test oracles to reveal merge conflicts. Then, we present our algorithm for selecting the UUTs that may contain merge conflicts from the the whole program. Finally, we detail the implementation of our tool TOM.

3.1. Test Oracles

3.1.1. Test Oracles for 3-way Merges

In our testing based approach, we need to find the inputs that make the contract of semantic conflict freedom fail. Before introducing our proposed test oracles, we first recall the algorithm (Sousa et al., 2018) used in SafeMerge for verifying the semantic conflict freedom on 3-way merges. As shown in Algorithm 1, after computing the post relation on output variables of four versions in a 3-way merge (Line 2), the algorithm validates whether the logic formula is satisfiable (Line 6).

Figure 3. Semantic conflicts that fail to be detected by SafeMerge.
1:function verify(,,,)
3:     )
4:     )
5:     )
6:     return
7:end function
Algorithm 1 Algorithm used in SafeMerge (Sousa et al., 2018) for verifying semantic conflict freedom on 3-way merges

By examining this algorithm, we find that it is inconsistent with the definition of semantic conflict freedom (Definition 2.1). For example, as shown in the Fig. 3, the method getX returns x based on the input flag. If the flag is set to true, the merge version returns the different value compared to all of its three ancestors (i.e., the original version and two variants). There is no doubt that this merge is not semantic conflict free according to the definition. However, this case would not be reported if we adopt Algorithm 1 to verify the property.444We have examined the implementation of SafeMerge, and the same problem exists. We revise the algorithm into the following one:

1:function verify(,,,)
3:     )
4:     )
5:     )
6:     return
7:end function
Algorithm 2 Revised algorithm for verifying conflict freedom

As shown in Algorithm 2, if there exists one input that makes the logic formula false (Line 6), we would say the conflict exists. To find the inputs revealing conflicts, we negate the , then we have , where


Hence, to reveal merge conflicts, we should find the inputs that make the formula

satisfiable, where represents the execution result on the program by inputs . In other words, after execution on four different versions with the same inputs , if is satisfiable, we say the inputs reveal the merge conflict. To simplify the problem of finding the inputs revealing conflicts, we give the following theorem to reformulate .

Theorem 1 ().

, where









Hence, we have , based on the above equation. ∎

The transformation guarantees that these three cases cover all cases that violate the contract of semantic conflict freedom. Specifically, if (1) one variable’s value returned by the merge is different from the values returned by two variants (i.e., ), or (2) one variable’s value returned by one variant is different from that of the original and that of the merge (i.e., or ), the merge is not semantic conflict free.

3.1.2. Generalized Test Oracles for n-parent Merges

By exploring further , and , we can find the similarities between them. In general, for any , given one version , if we find that its output is different from those of two other variants and , we say is true. To find the conflicts in one 3-way merge, we just need to repeatedly find one output of one version is different from those of the other versions. And according to Theorem 1, the repeatable processes of finding outputs meet and respectively on different versions guarantee the quality of the final program merge.

Hence, based on the above observation, the framework of Algorithm 2 (i.e., the repeatable process) works for general n-parent merges. In a 2-way merge scenario consisting of two variants and that do not have the common original, we can not tell what behavior is newly introduced by any of them. However, we are able to investigate whether the merge has some new behavior that is not introduced by any of its two parents, just by examining the . As for the multiple variants merged in the octopus merge scenario, we just need to investigate each of the variants repeatedly, like for those two variants of 3-way merge by examining the and . Similarly, for the merge version in the octopus merge scenario, as shown in Fig. 4, we just need to compare those outputs returned by variants {} with that of the merge version . Now, we are able to tell whether semantic conflicts arise for all real-world merge scenarios.

Figure 4. is the merge version (i.e., the target program for generating tests) and [, , …, ] are the variants (that need to fail the tests).

Test case is a piece of code fragment including inputs and invoking UUTs. Generally, assertions are often used to guard the values of those variables declared in the test case. If we have one test case generated for the version with the assertions on the values of output variables, then we can run this test case on all variants to check the failure of assertions. If all variants fail on the same assertion, we tell that the behavior described by this assertion is missing in all variants. Overall, the problem of finding inputs revealing conflicts can be transformed into the test generation problem.

Definition 3.1 (Test Oracle on Program Merges) Given an n-parent merge, to find the merge conflicts, we generate tests that achieve the following goals:

(1) Unexpected Behavior: Suppose that one test case is generated for the merged program . We say that has some unexpected behavior, if for any parent version , the same assertion is violated;

(2) Lost Behavior: Suppose that parent versions have the common original . For one parent version , we say its newly introduced behavior is missing after merging, if one test case for fails on and over the same assertion .

As we can see, our proposed test oracles make it in an easier way to detect conflicts, since we just need to repeatedly find one test case generated for one program version that can kill all of the variants by the same assertion.

3.2. UUTs Selection

As changes of different branches may be made to different locations across the whole program, we need to identity those methods whose behaviors have been affected by all branches. For the general case shown in Fig. 4, we extract all sets of entities (i.e., fields and methods) that have different behavior between versions {, , …, }. Once changes are made to one method, two versions of this method may have different behaviors. And as shown in Fig. 2, the effects of changes also can be propagated into other unchanged locations. Hence, if this changed method is called in the body of another method, the caller method may behave differently.

Figure 5. The workflow of TOM.

In our paper, we extract those added and changed entities (i.e., fields and methods) and we do not consider the deleted class, field and method. If one deleted method is not called by any methods, it seems meaningless to analyze this method. If this deleted method was called by other methods and the developer has not deleted all of the invocations in other methods, the compiler would report errors. As a result, to analyze the impact of deleted methods, we just need to analyze the modified methods. The added entities should be included since these entities may be modified in other version pairs. Imagine that in one 3-way merge, one method appears in the second parent versions and the merge version , and these versions of this method are different. This method in the merge should be analyzed since its behavior may be changed by developers after merging successfully.

Given one version together with a set of variants , after identifying added and changed entities from version pairs {, , …, }, we propose an algorithm for selecting UUTs from that behave differently in all variants, as shown in Algorithm 3. In some cases, the number of candidate methods may be large. For example, if changes on the assignments of the fields are made to the constructor, the other methods which use the changed fields should be analyzed since we have to instantiate these methods before calling them. Hence, one parameter is used to limit the size of all UUTs and one other parameter is used to limit the dependency depth explored. The method-level dependency analysis is conducted on the target version to extract the dependency relationships between entities of the whole program (Line 2). We then extract added and changed fields as well as methods (Lines 3-8) by comparing each variant with the target version. During the exploration, we get more directly impacted entities by the already collected entities (Lines 11-19). Then, we compute the intersection between the sets of impacted entities from each version pair as the UUTs (Line 20). If we have a set of UUTs more than the number specified, we return the first UUTs to generate tests (Line 21). If not, we find more impacted entities by exploring more deeply (Line 10). If we fail to find the common impacted entities during the search (Lines 10-24), we generate test cases for those added and changed methods to ensure the program quality (Lines 25-26).

1:function select_UUTs(, , , )
3:      // is a list of sets of changed entities
4:      // is a list of sets of impacted entities
5:     for   do
8:     end for
10:     for  do
11:         for  do
13:              for  do
14:                  if  then
16:                  end if
17:              end for
19:         end for
21:         if  then
22:              return
23:         end if
24:     end for
25:     if  then
27:     end if
28:end function
Algorithm 3 UUTs Selection

3.3. Implementation

As shown in Fig. 5, we present the work flow of our tool TOM. Given one n-parent merge, we generate tests for {,,…,} in order, as shown in the left part of Fig. 5. Then, for each case, we select the UUTs and then generate tests to reveal the conflicts. We employ the dependency analysis tool depends555 to parse the whole program and generate the dependencies between different entities.

We employ the advanced test generation tool EvoSuite (Fraser and Arcuri, 2011)

to implement the the test generation for program merges. EvoSuite utilizes meta-heuristic search techniques to generate and optimize test suites with respect to different code coverage criteria without reporting any false alarms 

(Gross et al., 2012). EvoSuite generates test cases by different advanced techniques such as alternating variable method, random testing and dynamic symbolic execution.

During the search process of generating tests, coverage criteria are used to generate the high-quality test suit. As for the UUTs in our case, it may have the same code in all variants and the target version, while the different parts are located in other methods, classes or packages. Existing implemented coverage criteria in EvoSuite instruments the code of the UUT, and then generate coverage goals for the UUT. High coverage on the UUT still cannot guarantee that the different parts are covered. Hence, we implement the diff-line coverage criterion to guide the search to achieve coverage goals on different lines between two versions.

Considering that the execution is expensive, based on the extracted different lines, we determine whether to execute the generated test case on all different variants to detect conflicts. Given two program versions with the test case , if their execution results are different, we can tell that the different parts between those two versions must be covered. Hence, as shown in Fig. 5, if one generated test can not cover any different part between two versions, we do not need to re-execute the test on the other version. If the generated test can cover the different parts, we then execute it on all of the variants.

For each execution of the test case on different variants, EvoSuite is able to collect the values of those variables in the test case. Given any two executions, we are able to generate assertions on the variables that have different values to capture the behavior difference between two versions. After running all the variants, we extract all the assertions that appear in each execution comparison. These assertions are what we need to reveal the conflicts. If one test case triggers some exception for the target version, we leave it to developers since the exception may be not desired. For each statement that has exceptions thrown in the execution on the variant, we generate all assertions for this statement based on the execution on the target version to describe the different values and states.

If we have the same assertion generated by executions on all variants, we then check the stability of this test by rerunning the tests five times. If we find one stable test, we add it to the test list that will be provided to developers to examine the merge conflicts. As shown in Fig. 5, we can configure TOM to stop the unit test generation once one test case revealing conflicts is generated or the given resource has been consumed (e.g., time out).

4. Experimental Evaluation

In this section, we describe the details of constructing the benchmark MCon4J, and conduct experiments to evaluate the effectiveness of TOM.

4.1. Benchmark

Sousa et al. (Sousa et al., 2018)

collect a total of 52 merges from nine real-world open-source Java applications. In our evaluation, we do not reuse these merges to conduct the experiments, because (1) changes of different branches are made to the same method in these 3-way merges; (2) the false positives and the false negatives may exist in their results, we are not able to ensure the conflict exists without knowing the counterexamples revealing the conflict; and (3) our tool’s effectiveness on detecting semantic conflicts heavily depends on the unit test generation tool. Although unit test generation techniques have been extensively improved, the advanced tools still fail to generate the tests that achieve high coverage for real-world programs consisting of complex objects, structures and logics. Honestly, the static verification techniques can outperform the dynamic unit test generation techniques in some aspects such as the time costs. Given the above considerations, we do not conduct experiments on these 52 merges collected by Sousa et al 

(Sousa et al., 2018).

To evaluate our tool’s effectiveness on detecting semantic conflicts, we need one benchmark consisting of 3-way merges and octopus merges. And for each merge, we need to know whether the conflicts exist or not. Otherwise we cannot tell that the tool works well if the testing based tool fails to find the conflict. And considering that changes made to different parts also may bring conflicts, we need to evaluate our tool on those merges whose parents modify the different methods. However, unlike the merges with textual conflicts, the merges that have semantic conflicts are pretty difficult to be identified by examining the evolutionary history of real-world programs. Hence, we decide to construct conflict 3-way and octopus merges by leveraging real-world bug-fix activities.

Brun et al. (Brun et al., 2011) classifies merge conflicts into textual and higher-order conflicts. The higher-order conflicts arise when changes are semantically incompatible and cause compilation errors, test failures, etc. Based on this notion of merge conflict, we construct conflict merges by making bug-fix tests fail. If we have one branch created for fixing bugs, but the test case fails after merging other branches, we consider conflict exists. The remaining problem is how to create other branches. Just et al. (Just et al., 2014b) conduct one investigation of the coupling effect between real faults and the mutants that are generated by commonly used mutation operators. The results show the existence of a coupling effect for 73% of real faults. Hence, we decide to use the generated mutants as the other branches along with the bug-fix branch to construct 3-way and octopus merges to conduct experiments.

Construction of 3-way merges. We first execute the bug-fix test case on the buggy program to collect the covered lines that may include those of unchanged classes during the bug fixes. Then, we make use of the mutation tool major (Just, 2014) to generate mutants on those covered lines of the buggy program. For each buggy program , we have the bug-fixed version and the mutant version . Using the default recursive merge strategy of Git, we construct merges that may have semantic conflicts by merging the and . For each buggy program, we have a number of merges. Then, we execute the bug-fix test case on the merged program , we get one conflict merge if this test case fails.

Construction of octopus merges. For each constructed 3-way merge, we create another branch by randomly choose one mutant of the same class mutated in the . Then, we merge these three branches by using the default merge strategy of Git. Similarly, the bug-fix test case will fail on the merge program.

Just et al. (Just et al., 2014a) propose Defects4J which collects a total of 438 reproducible bugs666, the latest commit is 17a99e1 to facilitate other software engineering research. For example, Defects4J is widely used in different fields such as unit test generation (Shamshiri et al., 2015), automated program repair (Martinez et al., 2017). For some versions of the Mockito project, Defects4J fails to generate mutants777 The issue remains open and has not been resolved till the date that we submit this paper. And for some bugs collected in Defects4J, we fail to construct conflict 3-way merges by merging mutants to make the bug-fix test cases fail. Finally, we collect a total of 389 conflict three-way merges and 389 conflict octopus merges.

As shown in Table 1, we list the numbers of merges in each project. The column “DC” means that the mutate branch and the fix branch modify the different classes. The column “SC” means that two branches modify the different methods of the same class. The column “SM” means that two branches modify the same method.

Project Bugs Merges
DC SC SM Total
JFreeChart 26 18 6 1 25
Closure 176 168 3 0 171
Commons Lang 65 16 34 11 61
Commons Math 106 61 32 6 99
Mockito 38 6 0 0 6
Joda-Time 27 23 4 0 27
Total 438 292 79 18 389
Table 1. The numbers of 3-way merges and octopus merges.

4.2. RQ1: How many conflict 3-way merges are found by SafeMerge and Tom respectively?

SafeMerge. Once changes of branches are made to different methods, SafeMerge fails to verify whether conflicts exist. As shown in Table I, there are 18 three-way merges in which modifications are made to the same method. We run SafeMerge on these 18 merges to evaluate its effectiveness. However, SafeMerge fails to deal with 17 out of 18 merges by throwing the same error888This issue has been reported to the authors while it has not been fixed till the date that we submit this paper. We would update the results once the issue is resolved. which stops the verification procedure. As for the remaining merge, SafeMerge fails to return results after running for one hour, which is much greater than the average time cost (i.e., most of them are less than one second and the greatest one is 4.45s) as shown in (Sousa et al., 2018). Unfortunately, we fail to utilize SafeMerge to detect conflicts on all of those merges from MCon4J.

TOM. Recall that for selecting UUTs we limit the explored depth and the total number of UUTs. In our experiments, we set the explored depth to 5 and the number of UUTs to 3 respectively. We conduct two groups of experiments guided by different coverage criteria. For the first experiment, we only use the proposed diff-line coverage criterion. For the second experiment, we add more coverage criteria999The used criteria implemented in EvoSuite includes branch, cbranch, weakmutation, output, exception, method and methodnoexception. used in EvoSuite by default. Because the random operators existing in the search process, we run TOM on each 3-way merge three times to have a comprehensive view on the tool’s ability.

Figure 6. The distributions of coverage information.

As shown in Table 2, we list the total of 45 conflict merges detected by TOM. There are 42 and 40 conflict 3-way merges detected respectively by two groups of experiments. We fail to tell the multiple criteria work better than the diff-line criterion by examining these experimental results.

Id Type Diff-line Multi-criteria Id Type Diff-line Multi-criteria
#1 #2 #3 #1 #2 #3 #1 #2 #3 #1 #2 #3
Chart_4 DC Math_16 SC
Chart_5 DC Math_27 DC
Chart_8 SC Math_29 DC
Chart_11 SC Math_32 DC
Chart_14 DC Math_37 DC
Chart_16 SC Math_46 SC
Chart_21 DC Math_47 SC
Chart_24 SC Math_49 DC
Closure_19 DC Math_56 SC
Closure_173 DC Math_60 DC
Lang_19 DC Math_63 SC
Lang_39 SC Math_70 SC
Lang_41 SC Math_71 DC
Lang_45 DC Math_73 SC
Lang_47 SC Math_80 SC
Lang_60 SC Math_81 SC
Lang_65 SM Math_83 DC
Math_1 DC Math_92 SM
Math_2 DC Math_93 SC
Math_4 DC Math_95 SC
Math_9 SC Math_97 DC
Math_10 DC Time_9 DC
Math_11 DC
Table 2. Conflict three-way merges that detected by executing three times with different coverage criteria “Diff-line” and “Multi-criteria”. “” means the conflict merge fails to be detected. “” means the conflict merge is detected when generating test cases for the variant (i.e., the bug-fix version). “” means the conflict merge is detected when generating test cases for non- variants. “” means the conflict merge is detected when generating test cases for both and non- variants.

During the process of generating test cases for merges, if we can find one execution path cover the changes, we may detect the merge conflict. In other words, higher coverage does not always mean the conflicts can be found. However, higher coverage still improves the possibility of detecting conflicts. As shown in the Fig. 6, we present the achieved maximum coverage during the test case generation for each target variant (i.e., the bug fixed version) guided by the diff-line coverage. As we can see, TOM achieves lower coverage on the projects Closure and Mockito than the four other projects. The existing study (Shamshiri et al., 2015) also shows that EvoSuite achieves the low coverage when generates test cases for the Closure project (the Mockito project is not included in the Defects4J at that time). It seems that the relatively high coverage is achieved for the project Time, while there are more flaky tests generated according to our observations on the test generation process. And these flaky tests affects the coverage information collected during the unit test generation.

4.3. RQ2: How does TOM perform on those constructed octopus merges?

We construct octopus merges based on constructed 3-way merges. To answer this question, we adapt the same settings and use the set of multiple criteria described in RQ1. Then, we run TOM on each octopus merge three times.

As shown in Table 3, we show the details of experimental detection results on octopus merges. There are a total of 87 conflict octopus merges detected by TOM. Comparing those detected conflict 3-way merges, we find those octopus merges from 35 out of 45 cases whose 3-way conflict merges have been detected. A total of 52 conflict octopus merges from newly appeared cases are detected. We construct octopus merges by adding one mutated branch based on the constructed 3-way merge. With more mutated code changing program behaviors, it makes sense that TOM can find more conflict octopus merges than conflict 3-way merges.

Id Type #1 #2 #3 Id Type #1 #2 #3 Id Type #1 #2 #3
Chart_2 DC Closure_97 DC Math_31 DC
Chart_4 DC Closure_108 DC Math_32 DC
Chart_7 DC Closure_111 DC Math_37 DC
Chart_11 SC Closure_138 DC Math_39 DC
Chart_14 DC Closure_140 DC Math_46 SC
Chart_20 DC Closure_148 DC Math_47 SC
Chart_21 DC Closure_152 DC Math_49 DC
Chart_24 SC Closure_165 DC Math_52 DC
Chart_25 DC Lang_15 DC Math_56 SC
Closure_1 DC Lang_17 DC Math_60 DC
Closure_2 DC Lang_29 DC Math_63 SC
Closure_3 DC Lang_34 DC Math_70 SC
Closure_15 DC Lang_36 SC Math_71 DC
Closure_19 DC Lang_37 SC Math_73 SC
Closure_24 DC Lang_38 SC Math_80 SC
Closure_26 DC Lang_39 SC Math_81 SC
Closure_27 DC Lang_41 SC Math_83 DC
Closure_32 DC Lang_45 DC Math_85 DC
Closure_47 DC Lang_47 SC Math_87 DC
Closure_55 DC Lang_60 SC Math_88 DC
Closure_67 DC Lang_65 SM Math_92 SM
Closure_72 DC Math_1 DC Math_93 SC
Closure_78 DC Math_4 DC Math_95 SC
Closure_80 DC Math_9 SC Math_96 SC
Closure_81 DC Math_11 DC Math_102 DC
Closure_84 DC Math_16 SC Math_103 DC
Closure_89 DC Math_25 SC Math_104 SC
Closure_95 DC Math_27 DC Time_9 DC
Closure_96 DC Math_29 DC Time_20 DC
Table 3. Conflict octopus merges detected by executing three times with the multiple coverage criteria. The same symbols are explained in the caption of Table II.

4.4. Threats to Validity

The main threats to the validity of our results belong to the internal and external validity threat categories.

Internal validity threats correspond to the implementation of TOM and the relevant scripts. Although we have reviewed the implementation carefully, the bugs may still exist and threat to the validity of our results.

External validity threats correspond to the constructed benchmark MCon4J. We mutate the source code by a set of mutant operators to simulate the real-world changes. Even though mutants can be used as good resource for research. Our tool may fail to achieve the similarly good results for detecting conflicts on real-world merge scenarios. The parameters affect the UUTs selection, and the values used also may fail to work well for other merge scenarios. As a result, these parameters may be adjusted in other merge scenarios.

5. Discussion

In this section, we discuss merge conflict resolution, the application scenario of TOM and the potential techniques on improving TOM.

Merge Conflict Resolution. Besides the textual tools, other existing syntactical and structural tools (Mens, 2002)(Zhu and He, 2018) also aim to keep all changes introduced by different branches in the merge version. However, the reasonableness of keeping and combining all changes may be questionable according to the contract of semantic conflict freedom. When different changes are made to the same assignment statement, the values of the variables would be different. In this case, whatever developers have done to resolve conflicts, the values of this variable in different versions would violate the contract of semantic conflict freedom. Without knowing other requirements on the merge, it is unable to resolve the conflicts automatically. For example, to resolve conflicts automatically, Xing and Maruyama (Xing and Maruyama, 2019) develops one automated program repair tool which needs one test case representing contracts provided by developers. Existing empirical studies (Menezes et al., 2018) on the resolutions of merge conflicts also show that developers are much likely to simply choose one version as the merge result when conflict arises. Hence, instead of replacing the textual tools with fine-grained tools to generate merges automatically, we think that we should pay more attention on guaranteeing the software quality after merging.

Continuous Integration. Continuous integration service is widely used in the open-source community to automatically find the build or test failures. Considering the workflow of collaborative development and the relative expensive costs of generating test cases for detecting conflicts, we think integrating the unit test generation in continuous integration is practical. After rerunning the whole test-suite, the continuous integration service is able to collect the coverage information. Then, if no tests fail, it utilizes the existing test case that covers changed code to seed the unit test generation to accelerate the procedure of generating test cases to trigger the semantic conflicts. Given the resource limit or the coverage requirements, the continuous integrations service is helpful to give developers more confidence on the quality of the merged software.

Potential Improvement. Besides the ability of test generation tools, identifying those methods that may have latent conflicts also plays an important role in the test generation for detecting conflicts. In our paper, we select the UUTs for generating tests based on the explicit call and use dependencies between changed fields, constructors and methods. However, we may miss some dependencies between changed entities, which leads to the failure of detecting conflicts. Those dependencies include the co-change relationships that can be mined from the software evolutionary history, the documented API usages and other common contracts (e.g., developers should call the close method to free resource after invoking open method). In real-world merge scenarios, we can seed the existing test cases that represents some dependency between entities to generate tests. Still, the missing of dependencies between methods has an impact on the detection results.

During the procedure of selecting UUTs, we extract all of the changed entities. Note that some changes like refactoring may not change the behavior at all. To prove the behavioral equivalence between program versions, much works (Lahiri et al., 2010; Partush and Yahav, 2014; Churchill et al., 2019) have been done. However, considering the effectiveness, soundness and availability of these existing tools, we currently do not make use of them to rule out those change impacted methods that may not behave differently. In future, with the help of advanced tools that can precisely determine the different behavior introduced by changes, we can save some computing resource by filtering out those changed methods whose behaviors are not changed.

6. Related Work

In this section, we mainly describe the related works on software merging and regression testing.

6.1. Software Merging

Over decades, software merging has been extensively studied because of its important role in software maintenance and evolution. Ahmed et al. (Ahmed et al., 2017) study the relationship between code smells and merge conflicts, and results show merges contain more code smells when conflicts arise. Nearly twenty years ago, Mens (Mens, 2002) provided a comprehensive summary of excellent works in this field. Different merging techniques such as textual, syntactic, semantic, structural and operation-based merging have been proposed very early.

However, the situation that developers rely on textual merge tools to deal with their daily work has not been changed. Mckee et al. (McKee et al., 2017) conduct interviews of 10 software practitioners to understand their perspectives on merge conflicts and resolutions. According to the unmet needs of software practitioners, they suggest researchers and tool builders focus on program comprehension, history exploration, etc. Nishimura and Maruyama (Nishimura and Maruyama, 2016) present one tool that exploits the fine-grained edit history to assist developers to examine the merge conflicts.

For the last decade, some new ideas and trends have emerged. Considering the variety of program languages, semi-structural merging (Apel et al., 2011) aims to achieve the balance between generality and performance. Proactive or early detection (Brun et al., 2011)(Guimarães and Silva, 2012)(Nguyen et al., 2015) of conflicts is used to decrease the possibility of merging branches with conflicts. Providing a set of candidate conflict resolutions to developers is also helpful. Niu et al. (Nan Niu, 2013)

develop a tool scoreRec that recommends the conflict resolutions ordered by estimating the cost and benefit of resolving conflicts. Zhu and He 

(Zhu and He, 2018) propose an interactive approach that ranks the conflict resolutions generated via the version space algebra. Xing and Maruyama (Xing and Maruyama, 2019) introduce the automatic program repair techniques to resolve the merge conflicts by leveraging the existing test cases.

Sousa et al. (Sousa et al., 2018) propose the contract of semantic conflict freedom inspired from much earlier work (Horwitz et al., 1989)(Yang et al., 1990), and then propose the verification on three-way merges to increase developers’ confidence on the merge result with respect to the contract. In this paper, we propose the test oracle inspired from the semantic conflict freedom and make it applicable for all merge scenarios. Utilizing the state-of-the-art unit test generation tool, we can generate tests to reveal conflicts if any.

6.2. Regression Testing

Regression test selection and regression test generation are the major techniques trying to prevent regression faults effectively with the low cost. The cost of rerunning the whole test suite is growing with the size and complexity of the evolving software. Various regression test selection techniques have been proposed. Zhu et al. (Zhu et al., 2019) propose the framework for checking the regression test selection tool.

Different from selection, regression test generation aims to generate new test cases that can expose the behavioral differences between two versions, when existing test suites fail to expose difference. Taneja and Xie (Taneja and Xie, 2008) synthesize one driver that calls two versions of the target method and adds conditions comparing the return values. Then, they utilize existing tools to generate test cases that cover the different branches in the driver method. As Jin et al. (Jin et al., 2010) explain, the generated test cases may not reveal the regression faults while cover the changed parts. Hence, they develop BERT to generate test cases that cover different parts first, and then analyze the behavioral differences to reveal the regression faults.

Person et al. (Person et al., 2008) propose differential symbolic execution that leverages symbolic execution techniques to characterize the changes, without providing the inputs to execute the changed program. Person et al. (Person et al., 2011) propose directed incremental symbolic execution to find those path conditions affected by code changes. Taneja et al. (Taneja et al., 2011) develop eXpress as one search strategy for dynamic symbolic execution to prune out paths that cannot lead to any code regions and those paths through which a state infection cannot propagate to any observable output.

Xu et al. (Xu et al., 2010) propose directed test suite augmentation that identifies code affected by changes and existing test cases relevant to testing that code. Then the identified test cases are used to seed the concolic or genetic test case generation approach to create new test cases that execute the affected code.

Software merging acts as the important activity during the software evolution, whereas there is no tool aiming to generate test cases revealing conflicts after merging. In our paper, we implement TOM to generate test cases for 2-way, 3-way and octopus merges.

7. Conclusion

In this paper, we propose the general test oracles for real-world program merges including 2-way, 3-way and octopus merges. Based on our test oracles, we propose an approach of regression unit test generation for detecting semantic conflicts. On this basis, we implement a tool called TOM to automatically generate test cases to reveal merge conflicts. In addition, we design the benchmark MCon4J to support further studies on regression test generation for software merges. In our experiments, a total of 45 conflict 3-way merges and 87 conflict octopus merges are identified by our tool TOM, while the state-of-the-art verification based tool SafeMerge fails to work on MCon4J. The experimental results show that our regression unit test generation tool is useful and effective in guaranteeing the quality of real-world program merges.


  • (1)
  • Ahmed et al. (2017) Iftekhar Ahmed, Caius Brindescu, Umme Ayda Mannan, Carlos Jensen, and Anita Sarma. 2017. An Empirical Examination of the Relationship Between Code Smells and Merge Conflicts. In Proceedings of the 11th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM ’17). 58–67.
  • Apel et al. (2011) Sven Apel, Jörg Liebig, Benjamin Brandl, Christian Lengauer, and Christian Kästner. 2011. Semistructured Merge: Rethinking Merge in Revision Control Systems. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (ESEC/FSE ’11). 190–200.
  • Brun et al. (2011) Yuriy Brun, Reid Holmes, Michael D. Ernst, and David Notkin. 2011. Proactive Detection of Collaboration Conflicts. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (ESEC/FSE ’11). 168–178.
  • Churchill et al. (2019) Berkeley Churchill, Oded Padon, Rahul Sharma, and Alex Aiken. 2019. Semantic Program Alignment for Equivalence Checking. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2019). 1027–1040.
  • Fraser and Arcuri (2011) Gordon Fraser and Andrea Arcuri. 2011. EvoSuite: automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (ESEC/FSE ’11). 416–419.
  • Gross et al. (2012) Florian Gross, Gordon Fraser, and Andreas Zeller. 2012. Search-Based System Testing: High Coverage, No False Alarms. In Proceedings of the 2012 International Symposium on Software Testing and Analysis (ISSTA ’12). ACM, New York, NY, USA, 67–77.
  • Guimarães and Silva (2012) Mário Luís Guimarães and António Rito Silva. 2012. Improving Early Detection of Software Merge Conflicts. In Proceedings of the 34th International Conference on Software Engineering (ICSE ’12). 342–352.
  • Horwitz et al. (1989) Susan Horwitz, Jan Prins, and Thomas Reps. 1989. Integrating Noninterfering Versions of Programs. ACM Transactions on Programming Languages and Systems 11, 3 (July 1989), 345–387.
  • Jiang et al. (2017) Siyuan Jiang, Collin McMillan, and Raul Santelices. 2017. Do Programmers do Change Impact Analysis in Debugging? Empirical Software Engineering 22, 2 (01 Apr 2017), 631–669.
  • Jin et al. (2010) Wei Jin, Alessandro Orso, and Tao Xie. 2010. Automated Behavioral Regression Testing. In Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation (ICST ’10). 137–146.
  • Just (2014) René Just. 2014. The Major mutation framework: Efficient and scalable mutation analysis for Java. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA). San Jose, CA, USA, 433–436.
  • Just et al. (2014a) René Just, Darioush Jalali, and Michael D. Ernst. 2014a. Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (ISSTA 2014). 437–440.
  • Just et al. (2014b) René Just, Darioush Jalali, Laura Inozemtseva, Michael D. Ernst, Reid Holmes, and Gordon Fraser. 2014b. Are Mutants a Valid Substitute for Real Faults in Software Testing?. In Proceedings of the 22Nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2014). 654–665.
  • Kasi and Sarma (2013) Bakhtiar Khan Kasi and Anita Sarma. 2013. Cassandra: Proactive Conflict Minimization Through Optimized Task Scheduling. In Proceedings of the 2013 International Conference on Software Engineering (ICSE ’13). 732–741.
  • Lahiri et al. (2010) Shuvendu K. Lahiri, Kapil Vaswani, and C A. R. Hoare. 2010. Differential Static Analysis: Opportunities, Applications, and Challenges. In Proceedings of the FSE/SDP Workshop on Future of Software Engineering Research (FoSER ’10). 201–204.
  • Landman et al. (2017) Davy Landman, Alexander Serebrenik, and Jurgen J. Vinju. 2017. Challenges for Static Analysis of Java Reflection: Literature Review and Empirical Study. In Proceedings of the 39th International Conference on Software Engineering (ICSE ’17). 507–518.
  • Leßenich et al. (2017) Olaf Leßenich, Janet Siegmund, Sven Apel, Christian Kästner, and Claus Hunsen. 2017. Indicators for merge conflicts in the wild: survey and empirical study. Automated Software Engineering (09 Sep 2017).
  • Martinez et al. (2017) Matias Martinez, Thomas Durieux, Romain Sommerard, Jifeng Xuan, and Martin Monperrus. 2017. Automatic repair of real bugs in java: a large-scale experiment on the defects4j dataset. Empirical Software Engineering 22, 4 (01 Aug 2017), 1936–1964.
  • McKee et al. (2017) S. McKee, N. Nelson, A. Sarma, and D. Dig. 2017. Software Practitioner Perspectives on Merge Conflicts and Resolutions. In Proceedings of IEEE International Conference on Software Maintenance and Evolution (ICSME ’17). 467–478.
  • Menezes et al. (2018) G. G. L. Menezes, L. G. P. Murta, M. O. Barros, and A. Van Der Hoek. 2018. On the Nature of Merge Conflicts: a Study of 2,731 Open Source Java Projects Hosted by GitHub. IEEE Transactions on Software Engineering (2018), 1–1.
  • Mens (2002) Tom Mens. 2002. A state-of-the-art survey on software merging. IEEE transactions on software engineering 28, 5 (2002), 449–462.
  • Nan Niu (2013) Jing-Ru C. Cheng Sandeep Reddivari Nan Niu, Fangbo Yang. 2013. Conflict resolution support for parallel software development. IET Software 7 (February 2013), 1–11(10). Issue 1.
  • Nguyen et al. (2015) Hung Viet Nguyen, My Huu Nguyen, Son Cuu Dang, Christian Kästner, and Tien N. Nguyen. 2015. Detecting Semantic Merge Conflicts with Variability-aware Execution. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015). 926–929.
  • Nishimura and Maruyama (2016) Y. Nishimura and K. Maruyama. 2016. Supporting Merge Conflict Resolution by Using Fine-Grained Code Change History. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Vol. 1. 661–664.
  • Partush and Yahav (2014) Nimrod Partush and Eran Yahav. 2014. Abstract Semantic Differencing via Speculative Correlation. In Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications (OOPSLA ’14). 811–828.
  • Person et al. (2008) Suzette Person, Matthew B. Dwyer, Sebastian Elbaum, and Corina S. Pǎsǎreanu. 2008. Differential Symbolic Execution. In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT ’08/FSE-16). 226–237.
  • Person et al. (2011) Suzette Person, Guowei Yang, Neha Rungta, and Sarfraz Khurshid. 2011. Directed Incremental Symbolic Execution. In Proceedings of the 32Nd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’11). 504–515.
  • Shamshiri et al. (2015) Sina Shamshiri, Rene Just, José Miguel Rojas, Gordon Fraser, Phil McMinn, and Andrea Arcuri. 2015. Do Automatically Generated Unit Tests Find Real Faults? An Empirical Study of Effectiveness and Challenges. In Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 201–211.
  • Sousa et al. (2018) Marcelo Sousa, Isil Dillig, and Shuvendu K. Lahiri. 2018. Verified Three-way Program Merge. Proceedings of the ACM on Programming Languages 2, OOPSLA, Article 165 (Oct. 2018), 29 pages.
  • Taneja and Xie (2008) Kunal Taneja and Tao Xie. 2008. DiffGen: Automated Regression Unit-Test Generation. In Proceedings of the 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE ’08). 407–410.
  • Taneja et al. (2011) Kunal Taneja, Tao Xie, Nikolai Tillmann, and Jonathan de Halleux. 2011. eXpress: Guided Path Exploration for Efficient Regression Test Generation. In Proceedings of the 2011 International Symposium on Software Testing and Analysis (ISSTA ’11). 1–11.
  • Xing and Maruyama (2019) Xiaoqian Xing and Katsuhisa Maruyama. 2019. Automatic Software Merging using Automated Program Repair. In 2019 IEEE 1st International Workshop on Intelligent Bug Fixing (IBF). 11–16.
  • Xu et al. (2010) Zhihong Xu, Yunho Kim, Moonzoo Kim, Gregg Rothermel, and Myra B. Cohen. 2010. Directed Test Suite Augmentation: Techniques and Tradeoffs. In Proceedings of the Eighteenth ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE ’10). 257–266.
  • Yang et al. (1990) Wuu Yang, Susan Horwitz, and Thomas Reps. 1990. A Program Integration Algorithm That Accommodates Semantics-preserving Transformations. SIGSOFT Softw. Eng. Notes 15, 6 (Oct. 1990), 133–143.
  • Zhu et al. (2019) Chenguang Zhu, Owolabi Legunsen, August Shi, and Milos Gligoric. 2019. A Framework for Checking Regression Test Selection Tools. In Proceedings of the 41st International Conference on Software Engineering (ICSE ’19). 430–441.
  • Zhu and He (2018) Fengmin Zhu and Fei He. 2018. Conflict Resolution for Structured Merge via Version Space Algebra. Proc. ACM Program. Lang. 2, OOPSLA, Article 166 (Oct. 2018), 25 pages.