# Proving Unrealizability for Syntax-Guided Synthesis

Proving Unrealizability for Syntax-Guided Synthesis We consider the problem of automatically establishing that a given syntax-guided-synthesis (SyGuS) problem is unrealizable (i.e., has no solution). Existing techniques have quite limited ability to establish unrealizability for general SyGuS instances in which the grammar describing the search space contains infinitely many programs. By encoding the synthesis problem's grammar G as a nondeterministic program P_G, we reduce the unrealizability problem to a reachability problem such that, if a standard program-analysis tool can establish that a certain assertion in P_G always holds, then the synthesis problem is unrealizable. Our method can be used to augment any existing SyGus tool so that it can establish that a successfully synthesized program q is optimal with respect to some syntactic cost -- e.g., q has the fewest possible if-then-else operators. Using known techniques, grammar G can be automatically transformed to generate exactly all programs with lower cost than q -- e.g., fewer conditional expressions. Our algorithm can then be applied to show that the resulting synthesis problem is unrealizable. We implemented the proposed technique in a tool called NOPE. NOPE can prove unrealizability for 59/134 variants of existing linear-integer-arithmetic SyGus benchmarks, whereas all existing SyGus solvers lack the ability to prove that these benchmarks are unrealizable, and time out on them.

Comments

There are no comments yet.

## Authors

• 5 publications
• 2 publications
• 4 publications
• 16 publications
• 21 publications
• ### Exact and Approximate Methods for Proving Unrealizability of Syntax-Guided Synthesis Problems

We consider the problem of automatically establishing that a given synta...
04/02/2020 ∙ by Qinheping Hu, et al. ∙ 0

read it

• ### Decidable Synthesis of Programs with Uninterpreted Functions

We identify a decidable synthesis problem for a class of programs of unb...
10/22/2019 ∙ by Paul Krogmeier, et al. ∙ 0

read it

• ### Semantics-Guided Synthesis

This paper develops a new framework for program synthesis, called semant...
08/22/2020 ∙ by Jinwoo Kim, et al. ∙ 0

read it

• ### Gradient Descent over Metagrammars for Syntax-Guided Synthesis

The performance of a syntax-guided synthesis algorithm is highly depende...
07/13/2020 ∙ by Nicolas Chan, et al. ∙ 0

read it

• ### SynRG: Syntax Guided Synthesis of Expressions with Alternating Quantifiers

Program synthesis is the task of automatically generating expressions th...
07/20/2020 ∙ by Elizabeth Polgreen, et al. ∙ 0

read it

• ### Just-in-Time Learning for Bottom-Up Enumerative Synthesis

A key challenge in program synthesis is the astronomical size of the sea...
10/16/2020 ∙ by Shraddha Barke, et al. ∙ 0

read it

• ### SyGuS-Comp 2018: Results and Analysis

Syntax-guided synthesis (SyGuS) is the computational problem of finding ...
04/12/2019 ∙ by Rajeev Alur, et al. ∙ 0

read it

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

The goal of program synthesis is to find a program in some search space that meets a specification—e.g., satisfies a set of examples or a logical formula. Recently, a large family of synthesis problems has been unified into a framework called syntax-guided synthesis (SyGuS). A SyGuS problem is specified by a regular-tree grammar that describes the search space of programs, and a logical formula that constitutes the behavioral specification. Many synthesizers now support a specific format for SyGuS problems [1], and compete in annual synthesis competitions [2]. Thanks to these competitions, these solvers are now quite mature and are finding a wealth of applications [9].

Consider the SyGuS problem to synthesize a function that computes the maximum of two variables and , denoted by . The goal is to create —an expression-tree for —where is in the language of the following regular-tree grammar :

 Start ::=  % Plus(Start,Start)∣IfThenElse(BExpr,Start,Start)∣x∣y∣0∣1BExpr ::=  GreaterThan(Start,Start)∣Not(BExpr)∣And(BExpr,BExpr% )

and is valid, where denotes the meaning of , and

 ψmax2(f,x,y):=f(x,y)≥x∧f(x,y)≥y∧(f(x,y)=x∨f(x,y)=y).

SyGuS solvers can easily find a solution, such as

 e:=IfThenElse(GreaterThan(x,y),x,y).

Although many solvers can now find solutions efficiently to many SyGuS problems, there has been effectively no work on the much harder task of proving that a given SyGuS problem is unrealizable—i.e., it does not admit a solution. For example, consider the SyGuS problem , where is the more restricted grammar with if-then-else operators and conditions stripped out:

 Start ::=  % Plus(Start,Start)∣x∣y∣0∣1

This SyGuS problem does not have a solution, because no expression generated by meets the specification.111Grammar only generates terms that are equivalent to some linear function of and ; however, the maximum function cannot be described by a linear function. However, to the best of our knowledge, current SyGuS solvers cannot prove that such a SyGuS problem is unrealizable.222The synthesis problem presented above is one that is generated by a recent tool called QSyGuS, which extends SyGuS with quantitative syntactic objectives [10]. The advantage of using quantitative objectives in synthesis is that they can be used to produce higher-quality solutions—e.g., smaller, more readable, more efficient, etc. The synthesis problem arises from a QSyGuS problem in which the goal is to produce an expression that (i) satisfies the specification , and (ii) uses the smallest possible number of if-then-else operators. Existing SyGuS solvers can easily produce a solution that uses one if-then-else operator, but cannot prove that no better solution exists—i.e., is unrealizable.

A key property of the previous example is that the grammar is infinite. When such a SyGuS problem is realizable, any search technique that systematically explores the infinite search space of possible programs will eventually identify a solution to the synthesis problem. In contrast, proving that a problem is unrealizable requires showing that every program in the infinite search space fails to satisfy the specification. This problem is in general undecidable [6]. Although we cannot hope to have an algorithm for establishing unrealizability, the challenge is to find a technique that succeeds for the kinds of problems encountered in practice. Existing synthesizers can detect the absence of a solution in certain cases (e.g., because the grammar is finite, or is infinite but only generate a finite number of functionally distinct programs). However, in practice, as our experiments show, this ability is limited—no existing solver was able to show unrealizability for any of the examples considered in this paper.

In this paper, we present a technique for proving that a possibly infinite SyGuS problem is unrealizable. Our technique builds on two ideas.

1. We observe that unrealizability can often be proven using finitely many input examples. In IllustrativeExample, we show how the example discussed above can be proven to be unrealizable using four input examples—, , , and .

2. We devise a way to encode a SyGuS problem over a finite set of examples as a reachability problem in a recursive program . In particular, the program that we construct has an assertion that holds if and only the given SyGuS problem is unrealizable. Consequently, unrealizability can be proven by establishing that the assertion always holds. This property can often be established by a conventional program-analysis tool.

The encoding mentioned in EncodingOfGrammarAndExamples is non-trivial for three reasons. The following list explains each issue, and sketches how they are addressed

1) Infinitely many terms. We need to model the infinitely many terms generated by the grammar of a given synthesis problem .

To address this issue, we use non-determinism and recursion, and give an encoding such that (i) each non-deterministic path in the program corresponds to a possible expression that can generate, and (ii) for each expression that can generate, there is a path in . (There is an isomorphism between paths and the expression-trees of )

2) Nondeterminism. We need the computation performed along each path in to mimic the execution of expression . Because the program uses non-determinism, we need to make sure that, for a given path in the program , computational steps are carried out that mimic the evaluation of for each of the finitely many example inputs in .

We address this issue by threading the expression-evaluation computations associated with each example in through the same non-deterministic choices.

3) Complex Specifications. We need to handle specifications that allow for nested calls of the programs being synthesized.

For instance, consider the specification . To handle this specification, we introduce a new variable and rewrite the specification as . Because is now also used as an input to , we will thread both the computations of and through the non-deterministic recursive program.

Our work makes the following contributions:

• We reduce the SyGuS unrealizability problem to a reachability problem to which standard program-analysis tools can be applied (IllustrativeExample and verification).

• We observe that, for many SyGuS problems, unrealizability can be proven using finitely many input examples, and use this idea to apply the Counter-Example-Guided Inductive Synthesis (CEGIS) algorithm to the problem of proving unrealizability (cegis).

• We give an encoding of a SyGuS problem over a finite set of examples as a reachability problem in a nondeterministic recursive program , which has the following property: if a certain assertion in always holds, then the synthesis problem is unrealizable (verification).

• We implement our technique in a tool nope using the ESolver synthesizer [2] as the SyGuS solver and the SeaHorn tool [8] for checking reachability. nope is able to establish unrealizability for 59 out of 132 variants of benchmarks taken from the SyGuS competition. In particular, nope solves all benchmarks with no more than 15 productions in the grammar and requiring no more than 9 input examples for proving unrealizability. Existing SyGuS solvers lack the ability to prove that these benchmarks are unrealizable, and time out on them.

RelatedWork discusses related work. Some additional technical material, proofs, and full experimental results are given in AdditionalMaterialproofssupp-eval, respectively.

## 2 Illustrative Example

In this section, we illustrate the main components of our framework for establishing the unrealizability of a SyGuS problem.

Consider the SyGuS problem to synthesize a function that computes the maximum of two variables and , denoted by . The goal is to create —an expression-tree for —where is in the language of the following regular-tree grammar :

 Start ::=  % Plus(Start,Start)∣IfThenElse(BExpr,Start,Start)∣x∣y∣0∣1BExpr ::=  GreaterThan(Start,Start)∣Not(BExpr)∣And(BExpr,BExpr% )

and is valid, where denotes the meaning of , and

 ψmax2(f,x,y):=f(x,y)≥x∧f(x,y)≥y∧(f(x,y)=x∨f(x,y)=y).

SyGuS solvers can easily find a solution, such as

 e:=IfThenElse(GreaterThan(x,y),x,y).

Although many solvers can now find solutions efficiently to many SyGuS problems, there has been effectively no work on the much harder task of proving that a given SyGuS problem is unrealizable—i.e., it does not admit a solution. For example, consider the SyGuS problem , where is the more restricted grammar with if-then-else operators and conditions stripped out:

 Start ::=  % Plus(Start,Start)∣x∣y∣0∣1

This SyGuS problem does not have a solution, because no expression generated by meets the specification.333Grammar generates all linear functions of and , and hence generates an infinite number of functionally distinct programs; however, the maximum function cannot be described by a linear function. However, to the best of our knowledge, current SyGuS solvers cannot prove that such a SyGuS problem is unrealizable. As an example, we use the problem discussed in Introduction, and show how unrealizability can be proven using four input examples: , , , and .