Erlang Code Evolution Control (Use Cases)

02/12/2018 ∙ by David Insa, et al. ∙ 0

The main goal of this work is to show how SecEr can be used in different scenarios. Concretely, we demonstrate how a user can run SecEr to obtain reports about the behaviour preservation between versions as well as how a user can use SecEr to find the source of a discrepancy. The use cases presented are three: two completely different versions of the same program, an improvement in the performance of a function and a program where an error has been introduced. A complete description of the technique and the tool is available at [1] and [2].

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Use case 1: Happy numbers

In order to show the potential of the tool, we provide an example to compare two versions of an Erlang program that computes happy numbers. Concretely, these two versions are based on completely different algorithms. Therefore, this use case shows that SecEr is able to deal with such scenarios where the correspondence between versions is not direct. Both of them are taken from the Rosetta Code repository:111Consulted in this concrete version: http://rosettacode.org/mw/index.php?title=Happy_numbers&oldid=251560#Erlang

http://rosettacode.org/wiki/Happy_numbers#Erlang.

1-spec main(pos_integer(),pos_integer()) ->
2  [pos_integer()].
3main(N, M) ->
4  happy_list(N, M, []).
5
6happy_list(_, N, L) when length(L) =:= N ->
7  lists:reverse(L);
8happy_list(X, N, L) ->
9  Happy = is_happy(X),
10  if Happy ->
11    happy_list(X + 1, N, [X|L]);
12  true ->
13    happy_list(X + 1, N, L) end.
14
15is_happy(1) -> true;
16is_happy(4) -> false;
17is_happy(N) when N > 0 ->
18  N_As_Digits =
19    [Y - 48 ||
20    Y <- integer_to_list(N)],
21  is_happy(
22    lists:foldl(
23      fun(X, Sum) ->
24        (X * X) + Sum
25      end,
26      0,
27      N_As_Digits));
28is_happy(_) -> false.
Listing 1: happy0.erl
1is_happy(X, XS) ->
2  if
3    X == 1 -> true;
4    X < 1 -> false;
5    true ->
6      case member(X, XS) of
7        true -> false;
8        false ->
9          is_happy(sum(map(fun(Z) -> Z*Z end,
10            [Y - 48 || Y <- integer_to_list(X)])),
11            [X|XS])
12      end
13  end.
14happy(X, Top, XS) ->
15  if
16    length(XS) == Top -> sort(XS);
17    Happy = is_happy(X,[]),
18    true ->
19      case  Happy of
20        true -> happy(X + 1, Top, [X|XS]);
21        false -> happy(X + 1,Top, XS)
22      end
23  end.
24
25-spec main(pos_integer(),pos_integer()) ->
26  [pos_integer()].
27main(N, M) ->
28  happy(N, M, []).
Listing 2: happy1.erl
$ ./secer -f happy0.erl -li 1  -var Happy -oc 1
          -f happy1.erl -li 2 -var Happy -oc 1
          -funs [main/2] -to 15
Function: main/2
----------------------------
Generated test cases: 812
Both versions of the program generate identical traces for the defined points of interest
----------------------------
Listing 3: SecEr reports that no discrepancies exist

First of all, and before using SecEr, we have to slightly adapt these codes. Our implementation needs a common input function, but in the original happy0 module there is a function main/0, which just runs a concrete test. Therefore, we have replaced this function with main/2 (see Listing 1) to match it with the corresponding one in the happy1 module (Listing 2). Moreover, in the happy1 module, we have added the Happy variable222This variable addition will not be needed when SecEr allows to define expressions as POIs. (line 2), which stores the result of is_happy(X,[]). In both modules, we have added a type specification (represented with spec in Erlang) to obtain more useful test cases in less time333Note that depending on the ITC, the execution of these programs can loop forever. Although we run each execution of a ITC with a limited timeout, these useless ITCs are spending valuable time that could be used by more useful ITCs..

Listing 3 shows the SecEr’s output when comparing both implementations of the program using a timeout of 15 seconds. The POI for both modules is the Happy variable, which is evaluated many times per execution, returning a trace with more than one element. As we can see, the execution of both implementations behave identically with respect to the Happy variable in the 812 generated test cases.

In order to see the output of the tool when the behaviour of the two compared programs differ, we have introduced two different errors in the happy1 module of Listing 2.

Listing 4: SecEr reports discrepancies for the error inside function is_happy using variable Happy as POI. Function: main/2 ---------------------------- Generated test cases: 759 Mismatching test cases: 66 (8.69%)     POIs comparison:         + {{ ’happy0.erl’,1,{var, ’Happy’},1},            { ’happy1.erl’,2,{var, ’Happy’},1}}                  Unexpected trace value => 66 Errors                  Example call: main(2,6) ------ Detected Error ------ Call: main(2,6) Error Type: Unexpected trace value POI: ({ ’happy0.erl’,1,{var, ’Happy’},1}) trace: [false,false,false,false,false,true,false,false,true,false,false,true,false,false,false,false, false,true,false,false,false,true,false,false,false,false,true] POI: ({ ’happy1.erl’,2,{var, ’Happy’},1}) trace: [false,false,false,false,false,false,false,false,true,false,false,true,false,false,false,false, false,true,false,false,false,true,false,false,false,false,true,false,false,true] ---------------------------- Listing 5: SecEr reports discrepancies for the error inside function is_happy using variable X as POI. Function: main/2 ---------------------------- Generated test cases: 905 Mismatching test cases: 105 (11.6%)     POIs comparison:         + {{ ’happy0.erl’,1,{var, ’X’},1},            { ’happy1.erl’,2,{var, ’X’},1}}                  The second trace is longer => 105 Errors                  Example call: main(6,3) ------ Detected Error ------ Call: main(6,3) Error Type: The length of both traces differs POI: ({ ’happy0.erl’,1,{var, ’X’},1}) trace:          [6,7,8,9,10,11,12,13] POI: ({ ’happy1.erl’,2,{var, ’X’},1}) trace:          [6,7,8,9,10,11,12,13,14,15,16,17,18,19] ----------------------------
An error inside the is_happy function

The first error is introduced by replacing the whole line 2 with X < 10 -> false;. With this change, the behaviour of both programs differs. When the user runs SecEr using the previous POI, it produces the report shown in Listing 4. SecEr is reporting that the POI has been executed several times and in some of these executions the values of the POI differed. Concretely, according to the given example, the executions of ITC main(2,6) compute different values at the sixth time both POIs are executed. Given this report, the user can continue testing using other POIs in order to find the source of the discrepancy. Since variable Happy is the result of calling the same function in both codes, there are two possible sources of the discrepancy: either the arguments that can take different values during the execution (i.e. X) or the code executed by the called function (i.e. is_happy). Following this idea, the user could use argument X as the new POI. In this case, if both versions produce different traces for X, then the discrepancy source must be in the parts of the code responsible for the values taken by variable X. Otherwise, the argument X can be discarded as the source of the discrepancy, and the source must be in something executed by the is_happy function. Listing 5 shows the report provided by SecEr when selecting variable X as the POI. The reported discrepancy indicates that both traces are the same until a point in the execution were the version in Listing 2 continues producing values. This behaviour is the expected one because the result of is_happy has an influence in the number of times the call is executed. Therefore, the user can conclude that the arguments do not produce the discrepancy and the error is inside the is_happy function.

An error outside the is_happy function
Listing 6: SecEr reports discrepancies for the error outside function is_happy using variable Happy as POI. Function: main/2 ---------------------------- Generated test cases: 830 Mismatching test cases: 798 (96.14%)     POIs comparison:         + {{ ’happy0.erl’,1,{var, ’Happy’},1},            { ’happy1.erl’,2,{var, ’Happy’},1}}                  Unexpected trace value => 798 Errors                  Example call: main(66,6) ------ Detected Error ------ Call: main(66,6) Error Type: Unexpected trace value POI: ({ ’happy0.erl’,1,{var, ’Happy’},1}) trace: [false,false,true,false,true,false,false,false,false,false,false,false,false,true,false,false, true,false,false,false,true,false,false,false,false,true] POI: ({ ’happy1.erl’,2,{var, ’Happy’},1}) trace: [false,true,false,false,false,false,false,true,false,true,false,false,false,false,true,false, true,false,true] ---------------------------- Listing 7: SecEr reports discrepancies for the error outside function is_happy using variable X as POI. Function: main/2 ---------------------------- Generated test cases: 787 Mismatching test cases: 766 (97.33%)     POIs comparison:         + {{ ’happy0.erl’,1,{var, ’X’},1},            { ’happy1.erl’,2,{var, ’X’},1}}                  Unexpected trace value => 766 Errors                  Example call: main(63,3) ------ Detected Error ------ Call: main(63,3) Error Type: Unexpected trace value POI: ({ ’happy0.erl’,1,{var, ’X’},1}) trace:          [63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79] POI: ({ ’happy1.erl’,2,{var, ’X’},1}) trace:          [63,65,67,69,71,73,75,77,79,80,82,83,85,87,89,91] ----------------------------

The second error consists in the replacement of the whole line 2 with false -> happy(X + 2, Top, XS). The output of SecEr in this case is depicted in Listing 6. The discrepancy reported is exactly the same one than the one reported in the previous error. Therefore, the user could use here variable X as the next POI following the same idea explained above. Listing 7 shows the report of SecEr in this case. Here the error is different from the one obtained above. It indicates that the values computed for X differ between versions. This is a clear indicator that this error is not being produced by function is_happy and it comes from where the variable X obtains its values. There are only two places in the code where this occurs (apart from the initial call that according to the reported error seems to be correctly defined), so now it is easy for the user to spot the error using this information.

Original version  

1tokens(S, Seps) ->
2    Res = tokens1(S, Seps, []).
3tokens1([C|S], Seps, Toks) ->
4  case member(C, Seps) of
5    true ->
6      tokens1(S, Seps, Toks);
7    false ->
8      tokens2(S, Seps, Toks, [C])
9  end;
10tokens1([], _Seps, Toks) ->
11  reverse(Toks).
12tokens2([C|S], Seps, Toks, Cs) ->
13  case member(C, Seps) of
14    true ->
15      tokens1(S, Seps, [reverse(Cs)|Toks]);
16    false ->
17      tokens2(S, Seps, Toks, [C|Cs])
18  end;
19tokens2([], _Seps, Toks, Cs) ->
20    reverse([reverse(Cs)|Toks]).

Optimized version  

1tokens(S, Seps) ->
2  Res =  
3    case Seps of
4      [] ->
5        case S of
6          [] -> [];
7          [_|_] -> [S]
8        end;
9      [C] ->
10        Res1 = tokens_single_1(reverse(S), C, []);
11      [_|_] ->
12        Res2 = tokens_multiple_1(reverse(S), Seps, [])
13    end.
14tokens_single_1([Sep|S], Sep, Toks) ->
15  tokens_single_1(S, Sep, Toks);
16tokens_single_1([C|S], Sep, Toks) ->
17  tokens_single_2(S, Sep, Toks, [C]);
18tokens_single_1([], _, Toks) ->
19  Toks.
20tokens_single_2([Sep|S], Sep, Toks, Tok) ->
21  tokens_single_1(S, Sep, [Tok|Toks]);
22tokens_single_2([C|S], Sep, Toks, Tok) ->
23  tokens_single_2(S, Sep, Toks, [C|Tok]);
24tokens_single_2([], _Sep, Toks, Tok) ->
25  [Tok|Toks].
26tokens_multiple_1([C|S], Seps, Toks) ->
27  case member(C, Seps) of
28    true ->
29      tokens_multiple_1(S, Seps, Toks);
30    false ->
31      tokens_multiple_2(S, Seps, Toks, [C])
32  end;
33tokens_multiple_1([], _Seps, Toks) ->
34  Toks.
35tokens_multiple_2([C|S], Seps, Toks, Tok) ->
36  case member(C, Seps) of
37    true ->
38      tokens_multiple_1(S, Seps, [Tok|Toks]);
39    false ->
40      tokens_multiple_2(S, Seps, Toks, [C|Tok])
41  end;
42tokens_multiple_2([], _Seps, Toks, Tok) ->
43  [Tok|Toks].

 

Figure 1: string.erl (original and optimized versions)

As we can observe in both errors, it does not matter where the bug is located as long as the bug affects the values computed at the POI. As we have seen in the examples, another interesting feature of the tool arises when we have a POI that is evaluated several times during the execution of the program. In this case, SecEr allows us to differentiate between two kinds of errors: traces that differ in their number of elements where a trace is a prefix of the other one (Listing 5) and traces with the same values in different order or with different values (Listing 7).

2 Use case 2: An improvement of the string:tokens/2 function

In this case of study, we consider a real commit of the Erlang/OTP distribution that improved the performance of the string:tokens/2 function. Figure 1 shows the code of the original and the improved versions. The differences introduced in this commit can be consulted here:

The improvement performed in this commit consists in two main changes. The first one is a general improvement obtained by reversing the input string (the one that is going to be tokenized) at the beginning of the process. The second one improves the cases where the separator list has only one element. The algorithm uses two auxiliary functions in both cases, so its structure is kept between both versions. However, the optimized code defines two versions of these functions to cover the single-element list of separators and the rest of cases separately.

We can use SecEr to check whether the behaviour of both versions is the same. A good way to start is to check whether the results produced by function tokens/2 are the same. In order to do this, we have to slightly modify the codes by adding variable Res in line 1 of the original version and in line 1 of the improved version444Note that this modification is only needed because of the current limitations of the tool. However, the approach has not this limitation, as it allows to select any expression, not only variables., both in Figure 1. Then, this variable is used as a POI. This allow the user to check that both versions preserve the same behaviour (see Listing 8).

$ ./secer -f string0.erl -li 1 -var Res -oc 1
          -f string1.erl -li 1 -var Res -oc 1
          -funs [main/2] -to 15
Function: tokens/2
----------------------------
Generated test cases: 7044
Both versions of the program generate identical traces for the defined points of interest
----------------------------
Listing 8: SecEr reports that no discrepancies exist
Function: tokens/2
----------------------------
Generated test cases: 6576
Mismatching test cases: 5040 (76.64%)
    POIs comparison:
        + {{ ’string0.erl’,1,{var, ’Res’},1},
           { ’string1.erl’,1,{var, ’Res’},1}}
                 Unexpected trace value => 5040 Errors
                 Example call: tokens([12,4,5],[2,3,2,5,0,1])
------ Detected Error ------
Call: tokens([12,4,5],[2,3,2,5,0,1])
Error Type: Unexpected trace value
POI: ({ ’string0.erl’,1,{var, ’Res’},1}) trace:
         [[[12,4]]]
POI: ({ ’string1.erl’,1,{var, ’Res’},1}) trace:
         [[4,[12,4]]]
---------------------
Listing 9: SecEr reports discrepancies after modifying optimized string.erl

We can now consider a hypothetical scenario where an error was introduced in the aforementioned commit. Suppose that line 1 in Figure 1 (optimized version) is replaced by the following expression:

[C | tokens_multiple_2(S, Seps, Toks, [C])]

Function: tokens/2
----------------------------
Generated test cases: 6570
Mismatching test cases: 6132 (93.33%)
    POIs comparison:
        + {{ ’string0.erl’,1,{var, ’Res’},1},
           { ’string1.erl’,1,{var, ’Res1’},1}}
                 The second trace is empty => 6132 Errors
                 Example call: tokens([0,5,2],[])
------ Detected Error ------
Call: tokens([0,5,2],[])
Error Type: The second trace is empty
POI: ({ ’string0.erk’,1,{var, ’Res’},1}) trace:
         [[[0,5,2]]]
POI: ({ ’string1.erl’,1,{var, ’Res1’},1}) trace:
         []
----------------------------
Listing 10: SecEr report using variable Res1 as the POI for the optimized version.
Function: tokens/2
----------------------------
Generated test cases: 7134
Mismatching test cases: 6649 (93.2%)
    POIs comparison:
        + {{ ’string0.erl’,1,{var, ’Res’},1},
           { ’string1.erl’,1,{var, ’Res2’},1}}
                 The second trace is empty => 1348 Errors
                 Example call: tokens([2,5,6,6,0],[])
        + {{ ’string0.erl’,1,{var, ’Res’},1},
           { ’string1.erl’,1,{var, ’Res2’},1}}
                 Unexpected trace value => 5301 Errors
                 Example call: tokens([9,10,2],[61,4,2,12,0,21,4,31])
------ Detected Error ------
Call: tokens([2,5,6,6,0],[])
Error Type: The second trace is empty
POI: ({ ’string0.erl’,1,{var, ’Res’},1}) trace:
         [[[2,5,6,6,0]]]
POI: ({ ’string1.erl’,1,{var, ’Res2’},1}) trace:
         []
----------------------------
------ Detected Error ------
Call: tokens([9,10,2],[61,4,2,12,0,21,4,31])
Error Type: Unexpected trace value
POI: ({ ’string_old.erl’,1,{var, ’Res’},1}) trace:
         [[[9,10]]]
POI: ({ ’string_error1.erl’,1,{var, ’Res2’},1}) trace:
         [[10,[9,10]]]
----------------------------
Listing 11: SecEr report using variable Res2 as the POI for the optimized version.

In this scenario, SecEr reports that some of the traces differ (see Listing 9). The error indicates that the produced values are not the expected ones. As in this particular case the main modification is the specialization of function tokens1/3, analyze the differences between the new versions of this function seems a good way to continue searching discrepancies, i.e. tokens_single_1/3 and tokens_multiple_1/3. For doing this, we need to add new variables Res1 and Res2 in lines 1 and 1 of Figure 1 (optimized version). The report for both POIs is depicted in Listings 10 and 11, respectively. Let us first analyze the report in Listing 10. At first sight, one can think that we have spotted the function responsible of the discrepancy, but this is not true. What SecEr is reporting here is that for some cases, the optimized version has not produced any value. For instance, the call tokens([0,5,2],[]), that SecEr uses to exemplify the problem, is executed by line 1 of the optimized version, so it is not entering to the case branch that executes tokens_single_1/3. Similar errors arise when the separator list contains more than one element, executing in this case the branch in line 1. So this kind of errors are expected and are not considered to be a discrepancy. On the other hand, the report on Listing 11 shows two types of discrepancies. The first one has the same explanation of the one reported in Listing 10. However, the second one corresponds to a real source of discrepancy, since they are cases where tokens_multiple_1/3 is executed and produced different results than tokens1/3. Note that almost 75% (5301 out of 7134) of the ITCs produced by SecEr correspond to the relevant error. This is due to the priority given to the discrepant traces as it is explained in Section 3.3 of [2]. Listing 11 also reveals one interesting feature of SecEr, which is that it can report different types of discrepancies for a given pair of POIs and provide the user with an example of each of them.

3 Use case 3: Complex numbers

 

1-record(complex, {real, img}).
2
3calculate(AR, AI, BR, BI) ->
4    A = #complex{real=AR, img=AI},
5    B = #complex{real=BR, img=BI},
6    Sum = add (A, B),
7    Product = multiply (A, B),
8    Negation = negation (A),
9    Inversion = inverse (A),
10    Conjugate = conjugate (A),
11    {Sum, Product, Negation, Inversion, Conjugate}.
12
13add (A, B) ->
14    RealPart = A#complex.real + B#complex.real,
15    ImgPart = A#complex.img + B#complex.img,
16    #complex{real=RealPart, img=ImgPart}.
17
18multiply (A, B) ->
19    RealPart = (A#complex.real * B#complex.real)
20        - (A#complex.img * B#complex.img),
21    ImgPart = (A#complex.real * B#complex.img)
22        + (B#complex.real * A#complex.img),
23    #complex{real=RealPart, img=ImgPart}.
24
25negation (A) ->
26    #complex{real=-A#complex.real, img=-A#complex.img}.
27
28inverse (A) ->
29    C = conjugate (A),
30    % Mod = (A#complex.real * A#complex.real) 
31    %  + (A#complex.img * A#complex.img), %RIGHT 
32    Mod = (A#complex.real * A#complex.img) 
33        + (A#complex.img * A#complex.img), %WRONG 
34    RealPart = C#complex.real / Mod,
35    ImgPart = C#complex.img / Mod,
36    #complex{real=RealPart, img=ImgPart}.
37
38conjugate (A) ->
39    RealPart = A#complex.real,
40    ImgPart = -A#complex.img,
41    #complex{real=RealPart, img=ImgPart}.

 

Figure 2: complex1.erl

The following use case is extracted555It is, in turn, built using a module from the Rosetta Code repository:
http://rosettacode.org/wiki/Arithmetic/Complex
from the use-case database of EDD666https://github.com/tamarit/edd, an algorithmic debugger for Erlang:

$ ./secer -f complex0.erl -li 2  -var Negation -oc 1
          -f complex1.erl -li 2 -var Negation -oc 1
          -funs [main/2] -to 15
Function: calculate/4
----------------------------
Generated test cases: 6459
Both versions of the program generate identical traces for the defined points of interest
----------------------------
Listing 12: Both versions behave in the same way for variable Negation
Function: calculate/4
----------------------------
Generated test cases: 6963
Mismatching test cases: 6844 (98.29%)
    POIs comparison:
        + {{ ’complex0.erl’,2,{var,  ’Inversion’},1},
           { ’complex1.erl’,2,{var,  ’Inversion’},1}}
                 The second trace is empty => 170 Errors
                 Example call: calculate(-75,0,38,6)
        + {{ ’complex0.erl’,2,{var,  ’Inversion’},1},
           { ’complex1.erl’,2,{var,  ’Inversion’},1}}
                 Unexpected trace value => 6674 Errors
                 Example call: calculate(-23,11.147602744042517,-13.290018732881684,
                                 7.818912260066709)
------ Detected Error ------
Call: calculate(-75,0,38,6)
Error Type: The first trace is empty
POI: ({ ’complex1.erl’,2,{var,  ’Inversion’},1}) trace:
         [{complex,-0.013333333333333334,0.0}]
POI: ({ ’complex0.erl’,2,{var,  ’Inversion’},1}) trace:
         []
----------------------------
------ Detected Error ------
Call: calculate(-23,11.147602744042517,-13.290018732881684,7.818912260066709)
Error Type: Unexpected trace value
POI: ({ ’complex0.erl’,2,{var,  ’Inversion’},1}) trace:
         [{complex,-0.03520754596864929,-0.017064336350048604}]
POI: ({ ’complex1.erl’,2,{var,  ’Inversion’},1}) trace:
         [{complex,0.1740765027306382,0.08437111737014728}]
----------------------------
Listing 13: Discrepancies reported for variable Inversion
Function: calculate/4
----------------------------
Generated test cases: 7326
Mismatching test cases: 7192 (98.17%)
    POIs comparison:
        + {{ ’complex0.erl’,2,{var, ’RealPart’},1},
           { ’complex1.erl’,2,{var, ’RealPart’},1}}
                 The second trace is empty => 188 Errors
                 Example call: calculate(11,0,-2.769732522433009,-1.9949121299611243)
        + {{ ’complex0.erl’,2,{var, ’RealPart’},1},
           { ’complex1.erl’,2,{var, ’RealPart’},1}}
                 Unexpected trace value => 7004 Errors
                 Example call: calculate(-206.34436567989434,15.930666184706574,1.6000469126136903,
                                 6.147738016916419)
------ Detected Error ------
Call: calculate(11,0,-2.769732522433009,-1.9949121299611243)
Error Type: The second trace is empty
POI: ({ ’complex0.erl’,2,{var, ’RealPart’},1}) trace:
         [0.09090909090909091]
POI: ({ ’complex1.erl’,2,{var, ’RealPart’},1}) trace:
         []
----------------------------
------ Detected Error ------
Call: calculate(-206.34436567989434,15.930666184706574,1.6000469126136903,6.147738016916419)
Error Type: Unexpected trace value
POI: ({ ’complex0.erl’,2,{var, ’RealPart’},1}) trace:
         [-0.004817552514292979]
POI: ({ ’complex1.erl’,2,{var, ’RealPart’},1}) trace:
         [0.06802373692423881]
----------------------------
Listing 14: Discrepancies reported for variable RealPart
Listing 15: No discrepancies reported for variable C Function: calculate/4 ---------------------------- Generated test cases: 7272 Both versions of the program generate identical traces for the defined points of interest ---------------------------- Listing 16: Discrepancies reported for variable Mod Function: calculate/4 ---------------------------- Generated test cases: 7489 Mismatching test cases: 7352 (98.17%)     POIs comparison:         + {{ ’complex0.erl’,2,{var, ’Mod’},1},            { ’complex1.erl’,2,{var, ’Mod’},1}}                  Unexpected trace value => 6706 Errors                  Example call: calculate(24,-10.509620259997908,14,-18) ------ Detected Error ------ Call: calculate(24,-10.509620259997908,14,-18) Error Type: Unexpected trace value POI: ({ ’complex0.erl’,2,{var, ’Mod’},1}) trace:          [686.4521180093585] POI: ({ ’complex1.erl’,2,{var, ’Mod’},1}) trace:          [-141.77876823059128] ----------------------------

This use case is formed by two versions of the same module that are almost identical except that one of them has a bug. Figure 2 shows the code (as it is on GitHub) of the buggy version. The bug is in lines 2 and 2, while the correct code is obtained by replacing these lines with lines 2 and 2. In the rest of the section, we show how SecEr can be used to detect this type of bugs.

We can start our test by choosing as a POI each variable of the resulting tuple of function calculate/4 in line 2, one by one. We named complex0 and complex1, the correct and the buggy versions of the code, respectively. The report given by SecEr for all the variables is similar to the one depicted in Listing 12, which shows that both versions behave in the same way, except for variable Inversion for which SecEr indicates that some discrepancies exist as it is shown in Listing 13

. In fact, this listing is showing two kinds of errors. The first indicates the lack of traces for one of the executions (probably due to runtime errors) and the second references to the discrepancy of the computed values. Since variable

Inversion comes from the result of the call to function inverse/1 in line 2, there are two possible sources of the discrepancy: the argument (variable A in this case) or the definition of function inverse/1. Therefore, the second step could be to check whether both versions are using the same arguments by placing a POI in variable A. However, this variable is used in all the other calls of function calculate/4, so this means that A most likely is not the source of the discrepancy. Therefore, the user could decide to continue searching for the bug inside function inverse/1 using as a POI the variables that defines the result, i.e. RealPart and ImgPart, both in line 2. Listing 14 shows the report provided by SecEr for variable RealPart. At this point, it is not needed to rerun SecEr using as a POI variable ImgPart, because we have already found an error. In this concrete case, if SecEr is rerun, a similar report will be produced. So, the next step is to place a POI in one of the two variables that define the value of RealPart, i.e. variables C and Mod in line 2. The report for variable C is depicted in Listing 15, whereas for variable Mod is in Listing 16. Thanks to these last reports, the user already can easily find the source of the discrepancy in the assignment of variable Mod.

References