1. Introduction
Relational properties describe requirements on the interaction between multiple programs or different runs of the same program. Examples of relational properties include the following:

Equivalence: Are two programs observationally equivalent? (i.e., )

Inversion: Given two programs , are they inverses of each other? (i.e.,

Noninterference: Given a program with two types of inputs, namely low (public) input and high (secret) input , does produce the same output when run on the same low input but two different high inputs ? (i.e.,

Transitivity: Given a program that returns a boolean value, does obey transitivity? (i.e., )
Observe that the first two properties listed above relate different programs, while the latter two relate multiple runs of the same program.
Due to their importance in a wide range of application domains, relational properties have received significant attention from the program verification community. For example, prior papers propose novel program logics for verifying relational properties (Benton, 2004; Barthe et al., 2012b; Sousa and Dillig, 2016; Chen et al., 2017; Yang, 2007) or transform the relational verification problem to standard safety by constructing socalled product programs (Barthe et al., 2011, 2013, 2016).
In this paper, we consider the dual synthesis problem of relational verification. That is, given a relational specification relating runs of programs , our goal is to automatically synthesize programs that satisfy . This relational synthesis problem has a broad range of practical applications. For example, we can use relational synthesis to automatically generate comparators that provably satisfy certain correctness requirements such as transitivity and antisymmetry. As another example, we can use relational synthesis to solve the program inversion problem where the goal is to generate a program that is an inverse of another program (Hu and D’Antoni, 2017; Srivastava et al., 2011). Furthermore, since many automated repair techniques rely on program synthesis (Nguyen et al., 2013; Jobstmann et al., 2005), we believe that relational synthesis could also be useful for repairing programs that violate a relational property like noninterference.
Solving the relational synthesis problem introduces new challenges over its nonrelational counterpart due to the combinatorially larger search space. In particular, a naive algorithm that simply enumerates combinations of programs and then checks their correctness with respect to the relational specification is unlikely to scale. Instead, we need to design novel relational synthesis algorithms to efficiently search for tuples of programs that collectively satisfy the given specification.
We solve this challenge by introducing a novel synthesis algorithm that learns relational version spaces (RVS), a generalization of the notion of version space utilized in prior work (Lau et al., 2003; Gulwani, 2011; Wang et al., 2017). Similar to other synthesis algorithms based on counterexampleguided inductive synthesis (CEGIS) (SolarLezama, 2008), our method also alternates between inductive synthesis and verification; but the counterexamples returned by the verifier are relational in nature. Specifically, given a set of relational counterexamples, such as or , our inductive synthesizer compactly represents tuples of programs and efficiently searches for their implementations that satisfy all relational counterexamples.
In more detail, our relational version space learning algorithm is based on the novel concept of hierarchical finite tree automata (HFTA). Specifically, an HFTA is a hierarchical collection of finite tree automata (FTAs), where each individual FTA represents possible implementations of the different functions to be synthesized. Because relational counterexamples can refer to compositions of functions (e.g., ), the HFTA representation allows us to compose different FTAs according to the hierarchical structure of subterms in the relational counterexamples. Furthermore, our method constructs the HFTA in such a way that tuples of programs that do not satisfy the examples are rejected and therefore excluded from the search space. Thus, the HFTA representation allows us to compactly represent those programs that are consistent with the relational examples.
We have implemented the proposed relational synthesis algorithm in a tool called Relish ^{1}^{1}1Relish stands for RELatIonal SyntHesis. and evaluate it in the context of two different relational properties. First, we use Relish to automatically synthesize encoderdecoder pairs that are provably inverses of each other. Second, we use Relish to automatically generate comparators, which must satisfy three different relational properties, namely, antisymmetry, transitivity, and totality. Our evaluation shows that Relish can efficiently solve interesting benchmarks taken from previous literature and online forums. Our evaluation also shows that Relish significantly outperforms EUSolver, a generalpurpose synthesis tool that won the General Track of the most recent SyGuS competition for syntaxguided synthesis.
To summarize, this paper makes the following key contributions:

We introduce the relational synthesis problem and take a first step towards solving it.

We describe a relational version space learning algorithm based on the concept of hierarchical finite tree automata (HFTA).

We show how to construct HFTAs from relational examples expressed as ground formulas and describe an algorithm for finding the desired accepting runs.

We experimentally evaluate our approach in two different application domains and demonstrate the advantages of our relational version space learning approach over a stateoftheart synthesizer based on enumerative search.
2. Overview
Property  Relational specification 
Equivalence  
Commutativity  
Distributivity  
Associativity  
Antisymmetry 
In this section, we define the relational synthesis problem and give a few motivating examples.
2.1. Problem Statement
The input to our synthesis algorithm is a relational specification defined as follows:
Definition 2.1 (Relational specification).
A relational specification with functions is a firstorder sentence , where is a quantifierfree formula with uninterpreted functions .
Fig. 1 shows some familiar relational properties like distributivity and associativity as well as their corresponding specifications. Even though we refer to Definition 2.1 as a relational specification, observe that it allows us to express combinations of both relational and nonrelational properties. For instance, the specification imposes both a nonrelational property on (namely, that all of its outputs must be nonnegative) as well as the relational property that the outputs of and must always add up to zero.
Definition 2.2 (Relational program synthesis).
Given a set of function symbols , their corresponding domainspecific languages , and a relational specification , the relational program synthesis problem is to find an interpretation for such that:

For every function symbol , is a program in ’s DSL (i.e., ).

The interpretation satisfies the relational specification (i.e., ).
2.2. Motivating Examples
We now illustrate the practical relevance of the relational program synthesis problem through several realworld programming scenarios.
Example 2.3 (String encoders and decoders).
Consider a programmer who needs to implement a Base64 encoder encode(x) and its corresponding decoder decoder(x) for any Unicode string x. For example, according to Wikipedia ^{2}^{2}2https://en.wikipedia.org/wiki/Base64, the encoder should transform the string “Man” into “TWFu”, “Ma” into “TWE=”, and “M” to “TQ==”. In addition, applying the decoder to the encoded string should yield the original string. Implementing this encoder/decoder pair is a relational synthesis problem in which the relational specification is the following:
Here, the first part (i.e., the first line) of the specification gives three inputoutput examples for encode, and the second part states that decode must be the inverse of encode. Relish can automatically synthesize the correct Base64 encoder and decoder from this specification using a DSL targeted for this domain (see Section 6.2).
Example 2.4 (Comparators).
Consider a programmer who needs to implement a comparator for sorting an array of integers according to the number of occurrences of the number 5 ^{3}^{3}3https://stackoverflow.com/questions/19231727/sortarraybasedonnumberofcharacteroccurrences. Specifically, compare(x,y) should return 1 (resp. 1) if x (resp. y) contains less 5’s than y (resp. x), and ties should be broken based on the actual values of the integers. For instance, sorting the array [“24”,“15”,“55”,“101”,“555”] using this comparator should yield array [“24”,“101”,“15”,“55”,“555”]. Furthermore, since the comparator must define a total order, its implementation should satisfy reflexivity, antisymmetry, transitivity, and totality. The problem of generating a suitable compare method is again a relational synthesis problem and can be defined using the following specification:
A solution to this problem is given by the following implementation of compare:
where countChar function returns the number of occurrences of a character in the input string, and intCompare is the standard comparator on integers.
Example 2.5 (Equals and hashcode).
A common programming task is to implement equals and hashcode methods for a given class. These functions are closely related because equals(x,y)=true implies that the hash codes of x and y must be the same. Relational synthesis can be used to simultaneously generate implementations of equals and hashcode. For example, consider an ExperimentResults class that internally maintains an array of numbers where negative integers indicate an anomaly (i.e., failed experiment) and should be ignored when comparing the results of two experiments. For instance, the results [23.5,1,34.7] and [23.5,34.7] should be equal whereas [23.5,34.7] and [34.7,23.5] should not. The programmer can use a relational synthesizer to generate equals and hashcode implementations by providing the following specification:
A possible solution consists of the following pair of implementations of equals and hashcode:
Example 2.6 (Reducers).
MapReduce is a software framework for writing applications that process large datasets in parallel. Users of this framework need to implement both a mapper that maps input key/value pairs to intermediate ones as well as a reducer which transforms intermediate values that share a key to a smaller set of values. An important requirement in the MapReduce framework is that the reduce function must be associative. Now, consider the task of using the MapReduce framework to sum up sensor measurements obtained from an experiment. In particular, each sensor value is either a real number or None, and the final result should be None if any sensor value is None. In order to implement this functionality, the user needs to write a reducer that takes sensor values x, y and returns their sum if neither of them is None, or returns None otherwise. We can express this problem using the following relational specification:
The first part of the specification gives inputoutput examples to illustrate the desired functionality, and the latter part expresses the associativity requirement on reduce. A possible solution to this relational synthesis problem is given by the following implementation:
3. Preliminaries
Since the rest of this paper requires knowledge of finite tree automata and their use in program synthesis, we first briefly review some background material.
3.1. Finite Tree Automata
A tree automaton is a type of state machine that recognizes trees rather than strings. More formally, a (bottomup) finite tree automaton (FTA) is a tuple , where

is a finite set of states.

is an alphabet.

is a set of final states.

is a set of transitions (or rewrite rules).
Intuitively, a tree automaton recognizes a term (i.e., tree) if we can rewrite into a final state using the rewrite rules given by .
More formally, suppose we have a tree where is a set of nodes labeled with , is a set of edges, and is the root node. A run of an FTA on is a mapping compatible with (i.e., given node with label and children in the tree, the run can only map to if there is a transition in ). Run is said to be accepting if it maps the root node of to a final state, and a tree is accepted by an FTA if there exists an accepting run of on . The language recognized by is the set of all those trees that are accepted by .
Example 3.1 ().
Consider a finite tree automaton with states , alphabet , final states , and transitions :
Intuitively, the states of this FTA correspond to boolean constants (i.e., represent false and true respectively), and the transitions define the semantics of the boolean connectives and . Since the final state is , the FTA accepts all boolean formulas (containing only and ) that evaluate to true. For example, is accepted by , and the corresponding tree and its accepting run are shown in Fig. 2.
3.2. Examplebased Synthesis using FTAs
Since our relational synthesis algorithm leverages prior work on examplebased synthesis using FTAs (Wang et al., 2017, 2018a), we briefly review how FTAs can be used for program synthesis.
At a highlevel, the idea is to build an FTA that accepts exactly the ASTs of those DSL programs that are consistent with the given inputoutput examples. The states of this FTA correspond to concrete values, and the transitions are constructed using the DSL’s semantics. In particular, the FTA states correspond to output values of DSL programs on the input examples, and the final states of the FTA are determined by the given output examples. Once this FTA is constructed, the synthesis task boils down to finding an accepting run of the FTA. A key advantage of using FTAs for synthesis is to enable search space reduction by allowing sharing between programs that have the same inputoutput behavior.
In more detail, suppose we are given a set of inputoutput examples and a contextfree grammar describing the target domainspecific language. We assume that is of the form where and are the terminal and nonterminal symbols respectively, is a set of productions of the form , and is the topmost nonterminal (start symbol) in . We also assume that the program to be synthesized takes arguments .
Fig. 3 reviews the construction rules for a single example (Wang et al., 2018a). In particular, the Input rule creates initial states for input example . We then iteratively use the Prod rule to generate new states and transitions using the productions in grammar and the concrete semantics of the DSL constructs associated with the production. Finally, according to the Output rule, the only final state is where is the start symbol and is the output example . Observe that we can build an FTA for a set of inputoutput examples by constructing the FTA for each example individually and then taking their intersection using standard techniques (Comon, 1997).
Remark. Since the rules in Fig. 3 may generate an infinite set of states and transitions, the FTA is constructed by applying the Prod rule a finite number of times. The number of applications of the Prod rule is determined by a bound on the AST depth of the synthesized program.
Example 3.2 ().
Consider a very simple DSL specified by the following CFG, where the notation is shorthand for for a unary identity function id:
Suppose we want to find a program (of size at most 3) that is consistent with the inputoutput example . Using the rules from Fig. 3, we can construct with states , alphabet , final states , and transitions :
For example, the FTA contains the transition because the grammar contains a production and we have . Since it is sometimes convenient to visualize FTAs as hypergraphs, Fig. 4 shows a hypergraph representation of this FTA, using circles to indicate states, double circles to indicate final states, and labeled (hyper)edges to represent transitions. The transitions of nullary alphabet symbols (i.e. ) are omitted in the hypergraph for brevity. Note that the only two programs that are accepted by this FTA are and .
4. Hierarchical Finite Tree Automata
As mentioned in Section 1, our relational synthesis technique uses a version space learning algorithm that is based on the novel concept of hierarchical finite tree automata (HFTA). Thus, we first introduce HFTAs in this section before describing our relational synthesis algorithm.
A hierarchical finite tree automaton (HFTA) is a tree in which each node is annotated with a finite tree automaton (FTA). More formally, an HFTA is a tuple where

is a finite set of nodes.

is a mapping that maps each node to a finite tree automaton.

is the root node.

is a set of interFTA transitions.
Intuitively, an HFTA is a treestructured (or hierarchical) collection of FTAs, where corresponds to the edges of the tree and specifies how to transition from a state of a child FTA to a state of its parent. For instance, the lefthand side of Fig. 5 shows an HFTA, where the interFTA transitions (indicated by dashed lines) correspond to the edges and in the tree.
Just as FTAs accept trees, HFTAs accept hierarchical trees. Intuitively, as depicted in the righthand side of Fig. 5, a hierarchical tree organizes a collection of trees in a treestructured (i.e., hierarchical) manner. More formally, a hierarchical tree is of the form , where:

is a finite set of nodes.

is a mapping that maps each node to a tree.

is the root node.

is a set of edges.
We now define what it means for an HFTA to accept a hierarchical tree:
Definition 4.1 ().
A hierarchical tree is accepted by an HFTA iff:

For any node , the tree is accepted by FTA .

For any edge , there is an accepting run of FTA on tree and an accepting run of on such that there exists a unique leaf node where we have .
In other words, a hierarchical tree is accepted by an HFTA if the tree at each node of is accepted by the corresponding FTA of , and the runs of the individual FTAs can be “stitched” together according to the interFTA transitions of . The language of an HFTA , denoted , is the set of all hierarchical trees accepted by .
Example 4.2 ().
Consider the HFTA shown on the lefthand side of Fig. 5. This HFTA has nodes annotated with FTAs as follows:
where:
with transitions :
with transitions :
with transitions :
Finally, includes the following four transitions:
Intuitively, this HFTA accepts ground firstorder formulas of the form where (resp. ) is a ground term accepted by (resp. ) such that and are equal. For example, and are collectively accepted. However, and are not accepted because is not equal to . Fig. 5 shows a hierarchical tree that is accepted by and another one that is rejected by .
5. Relational Program Synthesis using HFTA
In this section, we describe the highlevel structure of our relational synthesis algorithm and explain its subprocedures in the following subsections.
Our toplevel synthesis algorithm is shown in Algorithm 1. The procedure Synthesize takes as input a tuple of function symbols , their corresponding contextfree grammars , and a relational specification . The output is a mapping from each function symbol to a program in grammar such that these programs collectively satisfy the given relational specification , or null if no such solution exists.
As mentioned in Section 1, our synthesis algorithm is based on the framework of counterexampleguided inductive synthesis (CEGIS) (SolarLezama, 2008). Specifically, given candidate programs , the verifier checks whether these programs satisfy the relational specification . If so, the CEGIS loop terminates with as a solution (line 6). Otherwise, we obtain a relational counterexample in the form of a ground formula and use it to strengthen the current ground relational specification (line 7). In particular, a relational counterexample can be obtained by instantiating the quantified variables in the relational specification with concrete inputs that violate . Then, in the inductive synthesis phase, the algorithm finds candidates that satisfy the new ground relational specification (lines 811) and repeats the aforementioned process. Initially, the candidate programs are chosen at random, and the ground relational specification is true.
Since the verification procedure is not a contribution of this paper, the rest of this section focuses on the inductive synthesis phase which proceeds in three steps:

Relaxation: Rather than directly taking as the inductive specification, our algorithm first constructs a relaxation of (line 8) by replacing each occurrence of function in with a fresh symbol (e.g., is converted to ). This strategy relaxes the requirement that different occurrences of a function symbol must have the same implementation and allows us to construct our relational version space in a compositional way.

HFTA construction: Given the relaxed specification , the next step is to construct an HFTA whose language consists of exactly those programs that satisfy (line 9). Our HFTA construction follows the recursive decomposition of and allows us to compose the individual version spaces of each subterm in a way that the final HFTA is consistent with .

Enforcing functional consistency: Since relaxes the original ground relational specification , the programs for different occurrences of the same function symbol in that are accepted by the HFTA from Step (2) may be different. Thus, in order to guarantee functional consistency, our algorithm looks for programs that are accepted by the HFTA where all occurrences of the same function symbol correspond to the same program (lines 1011).
The following subsections discuss each of these three steps in more detail.
5.1. Specification Relaxation
Given a ground relational specification and a set of functions to be synthesized, the Relax procedure generates a relaxed specification as well as a mapping from each function to a set of fresh function symbols that represent different occurrences of in . As mentioned earlier, this relaxation strategy allows us to build a relational version space in a compositional way by composing the individual version spaces for each subterm. In particular, constructing an FTA for a function symbol requires knowledge about the possible values of its arguments, which can introduce circular dependencies if we have multiple occurrences of (e.g., ). The relaxed specification, however, eliminates such circular dependencies and allows us to build a relational version space in a compositional way.
We present our specification relaxation procedure, Relax, in the form of inference rules shown in Fig. 6. Specifically, Fig. 6 uses judgments of the form , meaning that ground specification is transformed into and gives the mapping between the function symbols used in and . In particular, a mapping in indicates that function symbols in all correspond to the same function in . It is worthwhile to point out that there are no inference rules for variables in Fig. 6 because specification is a ground formula that does not contain any variables.
Since the relaxation procedure is pretty straightforward, we do not explain the rules in Fig. 6 in detail. At a high level, we recursively transform all subterms and combine the results using a special union operator defined as follows:
In the remainder of this paper, we also use the notation to denote the inverse of . That is, maps each function symbol in the relaxed specification to a unique function symbol in the original specification (i.e., ).
5.2. HFTA Construction
We now turn our attention to the relational version space learning algorithm using hierarchical finite tree automata. Given a relaxed relational specification , our algorithm builds an HFTA that recognizes exactly those programs that satisfy . Specifically, each node in corresponds to a subterm (or subformula) in . For instance, consider the subterm where and are functions to be synthesized. In this case, the HFTA contains a node for whose child represents the subterm . The interFTA transitions represent data flow from the children subterms to the parent term, and the FTA for each node is constructed according to the grammar of the toplevel constructor of the subterm represented by .
Fig. 7 describes HFTA construction in more detail using the judgment where describes the DSL for each function symbol ^{4}^{4}4If there is an interpreted function in the relational specification, we assume that a trivial grammar with a single production for that function is added to by default. and is the mapping constructed in Section 5.1. The meaning of this judgment is that, under CFGs and mapping , HFTA represents exactly those programs that are consistent with . In addition to the judgment , Fig. 7 also uses an auxiliary judgment of the form to construct the HFTAs for the subformulas and subterms in the specification .
Let us now consider the HFTA construction rules from Fig. 7 in more detail. The first rule, called Const, is the base case of our algorithm and builds an HFTA node for a constant . Specifically, we introduce a fresh node and construct a trivial FTA that accepts only constant .
The next rule, Func, shows how to build an HFTA for a function term . In this rule, we first recursively build the HFTAs for each subterm . Next, we use a procedure called BuildFTA to build an FTA for the function symbol that takes arguments . The BuildFTA procedure is shown in Fig. 8 and is a slight adaptation of the exampleguided FTA construction technique described in Section 3.2: Instead of assigning the input example to each argument , our BuildFTA procedure assigns possible values that may take based on the HFTA for subterm . Specifically, let denote the final states of the FTA associated with the root node of . If there is a final state in , this indicates that argument of may take value ; thus our BuildFTA procedure adds a state in the FTA for (see the Input rule in Fig. 8). The transitions and new states are built exactly as described in Section 3.2 (see the Prod rule of Fig. 8), and we mark every state as a final state if is the start symbol in ’s grammar. To avoid getting an FTA of infinite size, we impose a predefined bound on how many times the Prod rule can be applied when building the FTA. Using this FTA for and the HFTAs for ’s subterms, we now construct a bigger HFTA as follows: First, we introduce a new node for function and annotate it with . Second, we add appropriate interFTA transitions in to account for the binding of formal to actual parameters. Specifically, if is a final state of , we add a transition to . Observe that our HFTA is compositional in that there is a separate node for every subterm, and the interFTA transitions account for the flow of information from each subterm to its parent.
is a vector where
represents the initial state set for argument .The next rule, called Logical, shows how to build an HFTA for a subterm where op is either a relation constant (e.g., ) or a binary logical connective (e.g., ). As in the previous rule, we first recursively construct the HFTAs for the subterms and . Next, we introduce a fresh HFTA node for the subterm “” and construct its corresponding FTA using op’s semantics. The interFTA transitions between and the FTAs annotating the root nodes of and are constructed in the same way as in the previous rule. ^{5}^{5}5In fact, the Logical and Neg rules could be viewed as special cases of the Func rule where op and have a trivial grammar with a single production. However, we separate these rules for clarity of presentation. Since the Neg rule for negation is quite similar to the Logical rule, we elide its description in the interest of space.
The last rule, Final, in Fig. 7 simply assigns the final states of the constructed HFTA. Specifically, given specification , we first use the auxiliary judgment to construct the HFTA for . However, since we want to accept only those programs that satisfy , we change the final states of the FTA that annotates the root node of to be rather than .
Example 5.1 ().
Consider the relational specification , where the DSL for is and the DSL for is (Here, denote the inputs of and respectively). We now explain HFTA construction for this specification.

First, the relaxation procedure transforms the specification to a relaxed version and produces a mapping relating the symbols in .

Fig. 9 shows the HFTA constructed for where we view and as unary functions for ease of illustration. Specifically, by the Const rule of Fig. 7, we build two nodes and that correspond to two FTAs that only accept constants 1 and 2, respectively. This sets up the initial state set of FTAs corresponding to and , which results in two HFTA nodes and and their corresponding FTAs and constructed according to the Func rule. (Note that we build the FTAs using only two applications of the Prod rule.) Similarly, we build a node and FTA by taking the final states of as the initial state set of . Finally, the node along with its FTA is built by the Logical rule.

Fig. 9 also shows two hierarchical trees, and , that are accepted by the HFTA constructed above. However, note that only corresponds to a valid solution to the original synthesis problem because (1) in both and refer to the program , and (2) in these two occurrences (i.e., and ) of correspond to different programs.
Theorem 5.2 ().
(HFTA Soundness) Suppose according to the rules from Fig 7, and let be the function symbols used in the relaxed specification . Given a hierarchical tree that is accepted by , we have:

is a program that conforms to grammar

satisfy , i.e.
Proof.
See Appendix A. ∎
Theorem 5.3 ().
(HFTA Completeness) Let be a ground formula where every function symbol occurs exactly once, and suppose we have . If there are implementations of such that
Comments
There are no comments yet.