Abstract Interpretation on E-Graphs

03/17/2022
by   Samuel Coward, et al.
Imperial College London
Intel
0

Recent e-graph applications have typically considered concrete semantics of expressions, where the notion of equivalence stems from concrete interpretation of expressions. However, equivalences that hold over one interpretation may not hold in an alternative interpretation. Such an observation can be exploited. We consider the application of abstract interpretation to e-graphs, and show that within an e-graph, the lattice meet operation associated with the abstract domain has a natural interpretation for an e-class, leading to improved precision in over-approximation. In this extended abstract, we use Interval Arithmetic (IA) to illustrate this point.

READ FULL TEXT VIEW PDF

Authors

page 1

page 2

page 3

05/30/2022

Combining E-Graphs with Abstract Interpretation

E-graphs are a data structure that compactly represents equivalent expre...
08/16/2017

A fix-point characterization of Herbrand equivalence of expressions in data flow frameworks

The problem of determining Herbrand equivalence of terms at each program...
06/22/2022

Decoupling the ascending and descending phases in Abstract Interpretation

Abstract Interpretation approximates the semantics of a program by mimic...
09/25/2018

Skeletal Semantics and their Interpretations

Many meta-languages have been proposed for writing rule-based operationa...
01/10/2020

A computational interpretation of Zorn's lemma

We give a computational interpretation to an abstract instance of Zorn's...
03/29/2019

A Provable Defense for Deep Residual Networks

We present a training system, which can provably defend significantly la...
10/14/2018

Monads, partial evaluations, and rewriting

Monads can be interpreted as encoding formal expressions, or formal oper...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

Recent e-graph applications have typically considered concrete semantics of expressions, where the notion of equivalence stems from concrete interpretation of expressions (Panchekha et al., 2015; Wang et al., 2020). However, equivalences that hold over one interpretation may not hold in an alternative interpretation. Such an observation can be exploited. We consider the application of abstract interpretation to e-graphs, and show that within an e-graph, the lattice meet operation associated with the abstract domain has a natural interpretation for an e-class, leading to improved precision in over-approximation. In this extended abstract, we use Interval Arithmetic (IA) (Neumaier, 1991; Jaulin et al., 2002) to illustrate this point.

IA is commonly used to provide tight bounds on expressions or numerical program outputs. This is useful across numerical hardware/software design and verification, providing guarantees that exceptional behaviour is never encountered and enabling deeper optimizations. In IA, every numerical expression is associated with an (real or floating-point) interval rather than a single numerical value and for each operation in the arithmetic, the natural interval extension operation is associated, where we assume that all infimums and supremums exist:

(1)

Let denote the natural interval extension of an expression, produced by structural induction on the expression syntax.

A key limitation of IA is the dependency problem, as illustrated through the following example. For a standard interpretation gives:

(2)

Implementing IA using e-graphs, helps to mitigate the effect of the dependency problem as we shall see in §2.

Among existing tools using IA to obtain tight bounds (Martin-Dorel and Melquiond, 2016; Daumas et al., 2005), Gappa (Daumas and Melquiond, 2010), a tool for fixed and floating-point error analysis, is particularly relevant as it deploys a set of term rewrites in order to tackle the dependency problem.

2. Approach

We implement IA for real arithmetic expressions on top of the extensible egg library (Willsey et al., 2021) as an e-class analysis. Following a standard approach, we represent real intervals by pairs of floating-point values, conservatively approximating real operations by rounding away from zero, a technique known as ‘outwardly rounded IA’ (Moore et al., 2009; Kulisch, 1981), however our examples in this abstract are presented as real numbers for simplicity.

The key insight of our work is that expressions that are equivalent in the concrete interpretation, and hence can belong to the same e-class in an e-graph, may differ in their abstract interpretation. Despite this difference in abstract interpretation, the soundness of the two or more different abstract interpretations of concrete-equivalent expressions, implies that they may be combined via the meet operation associated with the abstract lattice (Cousot and Cousot, 1977), producing a more precise approximation.

A trivial example of this process derives from Eqn. 2. Consider the expression x - x, together with the rewrite rule . This implies that , where denotes concrete-equivalence. In an e-graph, an e-class for this expression would contain two nodes, corresponding to the equivalent expressions. The interval interpretation of these two expressions is and , respectively, and as a result we may conclude that both expressions lie in , where intersection is the meet operation of the interval lattice.

To describe interval propagation throughout the e-graph, define to be the set of e-classes, and the set of e-nodes contained in . With each e-class, we associate a pair of floating-point values to represent a real interval, which we denote .

Similarly interpret a -arity e-node of function with children classes , as:

(3)

via the natural interval extension of the function of the e-node , as per Eqn. 1. 0-arity e-nodes represent constants, associated with degenerate intervals containing a single value, or variables, accompanied by user specified intervals.

For acyclic e-graphs, it is trivial to propagate the known intervals upwards through the e-graph using Eqn. 3 together with the following novel tightening relationship, where meet is intersection for intervals.

(4)

Although described above in static terms for simplicity of exposition, it’s important to note that Eqn. 4 can be evaluated on the fly, as we discover further equivalences and grow the e-graph; these always make the approximation more precise, due to the monotonicity of the meet operation: just as the e-graph grows monotonically during construction, the associated abstract values within the e-graph will monotonically narrow, corresponding to more precise expression bounds. We also note that this property allows for computation with cyclic e-graphs, under which Eqns 3 and 4 form a fixpoint specification.

Figure 1 demonstrates a non-trivial dependency problem example resolved by e-graph IA (Moore et al., 2009). The over-approximation in the expression, , stems from the multiple occurrences of in the expression. By rewriting, a concrete-equivalent expression is discovered, in which only appears once, removing the dependency issue.

3. Interval Implementation

[Initial e-graph: ]  [Applying ]

Figure 1. Including intervals in the e-graph and rewriting to obtain tight bounds. recip is the reciprocal function.

For this work we use a set of 23 rewrites. The basic arithmetic rewrites are commutativity, associativity, distributivity, cancellation and idempotent operation reduction across addition, subtraction, multiplication and division. The set also contains conditional polynomial rewrites for factorising using the quadratic formula and completing the square. Lastly, we include two operator specific rewrites.

Using these rewrites we are able to tighten expression bounds on the simple expressions presented in Table 1. On these small examples egg runs in seconds, since interval calculations are relatively cheap.

Expression Initial Improved Width Change
[-2,3] [0,1] -80%
[-5,1.5] [-4.5, 0] -31%
-93%
-33%
Table 1. Interval tightening via e-graphs, for . Width change=(improved width - initial width)/initial width.

To demonstrate an advantage of this approach, consider the following example, for , where the following concrete-equivalences are discovered via rewriting:

(5)
(6)
(7)

The interval associated with the e-class containing these three expressions is . We observe that there is no need to find a single expression providing both bounds, which may in general be impossible. In more general abstract interpretations, we may find a set of expressions, each providing valuable and distinct information.

4. Conclusion and Future Work

We presented the application of abstract interpretations to e-graphs, which has two key advantages. Due to constructive rewrite application deciding which rewrites to apply and in which order is not a concern in the e-graph, useful if the route to tightly bounding expressions is non-obvious. There is also no constraint on the number of expressions that can provide relevant information in a given interpretation.

Further work will explore more complex problems, along with comparisons against existing tools such as Gappa (Daumas and Melquiond, 2010), and results on relational domains. We will apply rewrite rule inference (Nandi et al., 2021) to explore the space of bound tightening rewrites. It may be further possible to exploit cyclic e-graphs on abstract domains by interpreting the fixpoint equations as defining an iterative numerical method such as the Krawczyk method (Moore et al., 2009) which may then be extracted from the e-graph. Incorporating the technique into tools, such as Herbie (Panchekha et al., 2015), where bounds can be exploited, would demonstrate its value.

References

  • P. Cousot and R. Cousot (1977) Abstract interpretation: ”A” unified lattice model for static analysis of programs by construction or approximation of fixpoints. In Conference Record of the Annual ACM Symposium on Principles of Programming Languages, Vol. Part F130756. External Links: Document, ISSN 07308566 Cited by: §2.
  • M. Daumas, G. Melquiond, and C. Muñoz (2005) Guaranteed proofs using interval arithmetic. In Proceedings - Symposium on Computer Arithmetic, External Links: Document Cited by: §1.
  • M. Daumas and G. Melquiond (2010) Certification of bounds on expressions involving rounded operators. ACM Transactions on Mathematical Software 37 (1). External Links: Document, ISSN 00983500 Cited by: §1, §4.
  • L. Jaulin, M. Kieffer, O. Didrit, and É. Walter (2002) Applied interval analysis. Choice Reviews Online 39 (06). External Links: Document, ISSN 0009-4978 Cited by: §1.
  • U. Kulisch (1981) Computer Arithmetic in Theory and Practice. External Links: Document Cited by: §2.
  • É. Martin-Dorel and G. Melquiond (2016) Proving Tight Bounds on Univariate Expressions with Elementary Functions in Coq.

    Journal of Automated Reasoning

    57 (3).
    External Links: Document, ISSN 15730670 Cited by: §1.
  • R. E. Moore, R. B. Kearfott, and M. J. Cloud (2009) Introduction to Interval Analysis. External Links: Document Cited by: §2, §2, §4.
  • C. Nandi, M. Willsey, A. Zhu, Y. R. Wang, B. Saiki, A. Anderson, A. Schulz, D. Grossman, and Z. Tatlock (2021) Rewrite rule inference using equality saturation. Proceedings of the ACM on Programming Languages 5 (OOPSLA). External Links: Document, ISSN 24751421 Cited by: §4.
  • A. Neumaier (1991) Interval Methods for Systems of Equations. External Links: Document Cited by: §1.
  • P. Panchekha, A. Sanchez-Stern, J. R. Wilcox, and Z. Tatlock (2015) Automatically improving accuracy for floating point expressions. ACM SIGPLAN Notices 50 (6), pp. 1–11. Cited by: §1, §4.
  • Y. R. Wang, S. Hutchison, J. Leang, B. Howe, and D. Suciu (2020) SPORES: Sum-Product Optimization via Relational Equality Saturation for Large Scale Linear Algebra. External Links: Link Cited by: §1.
  • M. Willsey, C. Nandi, Y. R. Wang, O. Flatt, Z. Tatlock, and P. Panchekha (2021) Egg: Fast and extensible equality saturation. Proceedings of the ACM on Programming Languages 5 (POPL), pp. 1–29. Cited by: §2.