Towards an Algebraic Theory of Analogical Reasoning in Logic Programming

09/26/2018 ∙ by Christian Antic, et al. ∙ Apple, Inc. 0

Analogy-making is an essential part of human intelligence and creativity. This paper proposes an algebraic model of analogical reasoning in logic programming based on the syntactic composition and decomposition of programs. The main idea is to define analogy in terms of modularity and to derive abstract forms of concrete programs from a `known' source domain which can then be instantiated in an `unknown' target domain to obtain analogous programs. To this end, we introduce algebraic operations for program modularity and illustrate, by giving numerous examples, that programs have nice decompositions. Interestingly, our work suggests a close relationship between modularity, generalization, and analogy which we believe should be explored further in the future. In a broader sense, this paper is a first step towards an algebraic (and mainly syntactic) theory of analogical reasoning in logic-based knowledge representation and reasoning systems, with potential applications to fundamental AI-problems like computational learning and creativity.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Analogy-making is an essential part of human intelligence and creativity, with applications to such diverse tasks as commonsense reasoning, problem solving, and story telling (cf. [12, 7]).

This paper proposes an algebraic model of analogical reasoning in logic programming based on the syntactic composition and decomposition of programs. The main idea is to define analogy in terms of modularity and to derive abstract forms of concrete programs from a ‘known’ source domain which can then be instantiated in a ‘novel’ target domain to obtain analogous programs. To this end, we introduce algebraic operations for program modularity and illustrate, by giving numerous examples, that programs have nice decompositions.

In general, ‘creative’ analogies contain some form of unexpectedness, in the sense that objects (possibly from different domains) are put together in a novel and often unforeseen way. Consider the following running example.

Example 1

Imagine two domains, one consisting of numerals of the form , , , , formally representing the natural numbers, and the other made up of lists and so on. We know from basic arithmetic what it means to add two numbers (i.e., numerals), e.g., expresses the fact that one plus one equals two. Now suppose we want to transfer the concept of addition to the list domain. We can then ask—by analogy—the following question: What does it mean to ‘add’ two lists? There are (at least) two possible answers: One is to say that a list , say, is the ‘addition’ of two lists and , say, if the number of elements in equals the sum of the number of elements in and . In this case, we do not take the inner structure111One can distinguish between different elements of a list. of lists into account and we directly transfer the concept of number to lists, e.g., in analogy to , the fact is perfectly derivable under this interpretation. However, if we do respect the inner structure of lists, we obtain a second interpretation in which is the ‘addition’ of and if is the list appended by . In this case, we have specialized the analogy to the concrete domain at hand.222Note that we still have the property that the length of the list is the sum of the lengths of and . This simple example shows the inexact nature of analogies.

We can now ask a further question: Given that we ‘know’ how to add numbers, can we use this knowledge to answer questions in the list domain? It is evident that in the first case, where we only cared about the number of elements in a list, we can simulate ‘addition’ of lists via addition of numerals. The more interesting case is the second one, that is, given that multiple lists have the same length and thus map to the same numeral, can we still ‘add’ (i.e., append) lists via the (seemingly) simpler addition of numerals?333What we mean here is a simple mapping between lists and numerals which makes no use of any complicated form of encoding. Surprisingly, the answer is again ‘yes’, and we will return to this question, in a more formal way, in Example exa:Plus and 8 (in which we also discuss the important role of (un)typing for analogical reasoning).

The rest of the paper is structured as follows. The next section is preliminary and recalls the syntax and proof theory of an untyped version of higher-order Horn logic programming. sec:PR introduces two algebraic operations for program modularity, namely ‘composition’ and ‘concatenation’, on which the notions of analogy and logic program forms will be based. The following Sections 4 and 5 constitute the main part of the paper. In sec:Analogy we define the notion of analogy and show how analogies can be used to transfer knowledge across different domains by formally and precisely discussing Example exa:exa. After that we introduce in sec:Forms the notion of logic program form and show that forms subsume analogies. We then sketch, by giving detailed examples, a procedure for analogical reasoning via decomposition, abstraction, and specialization. Finally, we conclude the paper with a discussion and some ideas for future research.

For excellent surveys on computational models of analogical reasoning we refer the reader to [8, 15, 16].

2 Preliminaries

We recall the syntax and proof theory of higher-order Horn logic programs where we restrict ourselves to the untyped case.

2.1 Syntax

Let be an unranked nonempty alphabet whose elements are called symbols. Terms are formed as usual from variables and symbols from , where constants are treated as 0-ary symbols. A term is ground if it contains no variables. The set of ground terms is called the Herbrand universe of and is denoted by .

An (untyped higher-order Horn logic) program over (or -program) is a set of rules of the form

(1)

where are terms. A program is called ground if each of its rule is ground. It will be convenient to define, for a rule of the form (equ:r), (head) and (body) extended to programs by and . A fact is a rule with empty body, whereas a proper rule is a rule which is not a fact. We denote the facts (resp., proper rules) of by (resp., ). We will occasionally make some of the variables occurring in a program explicit by writing, e.g., for the program containing the variables .

Substitutions and unifiers are defined as usual and we denote the most general unifier of two terms by . Substitutions operate on expressions as usual and we denote the application of a substitution on an expression by .

For a symbol and a rule of the form (equ:r), we define the wrapping of via by

extended to programs by . The reverse of a program is obtained by reversing all the arrows in , that is,

A multiset of rules is a submultiset of , denoted , if consists only of rules from .

2.2 SLD-Resolution

Logic programs compute via a restricted form of resolution, called SLD-resolution, as follows.

For simplicity, we consider here only the ground case. Let be a ground query , , and suppose that for some , and , is a rule from . Then given by

is called a resolvent of and , and is called the selected term of . By iterating this process we obtain a sequence of resolvents which is called an SLD-derivation. A derivation can be finite or infinite. If its last query is empty then we speak of an SLD-refutation of the original query . In this case we have derived an SLD-proof of . A failed SLD-derivation is a finite derivation which is not a refutation. In case is a ground term with an SLD-proof from , we say that is an SLD-consequence of and write . Given two programs we say that is SLD-equivalent to if implies and vice versa, for all ground terms . For a rule of the form (equ:r), we write in case whenever holds for every . We denote the empty query by .

A type program for (or -type program) is a program which ‘generates’ the Herbrand universe of in the sense that any ground term is in if, and only if, .

Example 2

In this paper, we will be interested in three basic data structures: numerals, lists, and (binary) trees. The corresponding type programs are constructed as follows. Let , , and be alphabets. As is customary in logic programming, we abbreviate lists by writing, e.g., instead of and so on. Then, we define the -, -, and -type programs and , respectively, by

3 Modularity

In the rest of the paper, and will denote logic programs over some common alphabet if not specified otherwise.

This section introduces two (syntactic) composition operations on which we will base our notions of analogy and logic program forms.

Definition 1

We define the composition of and by

In the sequel, we simply write if the composition operation is understood. We denote the neutral element with respect to composition by 1.

Roughly, we obtain the composition of and by resolving all body atoms in with the ‘matching’ rule heads of . Composition is thus related to SLD-resolution in the sense that we can answer queries across different domains via decomposition (cf. Example exa:Plus).

Example 3

Reconsider the program of Example exa:omega_ generating the natural numbers. By composing the only proper rule in with itself, we obtain

Note that this program, together with the single fact in , generates the even numbers. Let us therefore define the program

(2)

We will come back to this example in (sec:Forms) when studying logic program forms.

In some cases, a program is the ‘concatenation’ of two other programs on an atomic level. A typical example is the program

which is, roughly, the ‘concatenation’ of the -type program in the first argument and the -type program in the second argument wrapped by the symbol (cf. Example exa:omega_). This is formalized as follows.

Definition 2

We define the concatenation of and inductively as follows:

  1. For symbols and sequences of terms ,

  2. For rules ,

  3. Finally, the concatenation of and is given by

We can now formally decompose the program from above with respect to concatenation by

We will return to decompositions of this form in sec:Forms.

4 Analogy

In this section, we formalize the concept of two programs, possibly from different domains, being syntactically analogous (or similar).

Definition 3

We say that can be (one-step) simulated by , in symbols , if there are programs over such that . In this case, we call the programs the left and right transfer programs, respectively. If and , we say that and are analogous (or similar) and write .

In other words, can be simulated by if there is a decomposition of (over the joint alphabet ) in which occurs as a factor, that is, if can be algebraically constructed from . The intuition is that the transfer programs in the decomposition of function as ‘bridges’ for transferring information between and , as is illustrated by the following example.

Example 4

To return to our introductory Example exa:exa, consider the well-known program

implementing the addition of positive numbers (represented as numerals). We can now ask—by analogy—what it means to ‘add’ two lists. As we have informally discussed in Example exa:exa, there are at least two possible answers. The first is to say that a list , say, is the ‘addition’ of the lists , say, if the number of elements of equals the sum of the number of elements in and . This solution is represented by the program

Note that the program does not take the inner structure of lists into account; e.g., we can derive

The program is arguably the ‘freest’ translation of into the list domain in the sense that for any rule consisting only of shared symbols (e.g., ), we have

(3)

For example, the rule

expressing commutativity of addition, is derivable from both programs. However, if we do take the inner structure of lists into account, we obtain a second solution where is the ‘addition’ of and if is appended by . This case is represented by the program

In a sense, we can say that is a specialization of to the concrete list domain. As a consequence, does not satisfy (equ:Iff).

We now want to show that all of the three programs in this example are analogous according to Definition def:analogy. We first show that and are analogous. For this, we define the transfer program by

and compute

We now show that and are analogous by defining the transfer program

and by computing

This shows

Finally, we want to mention that there are many other (list) programs analogous to (with different ‘bridges’, of course), which shows the inexact nature of analogy (see the discussion in sec:Discussion).

The following SLD-derivation demonstrates how we can use the transfer program from above to append two lists via the seemingly simpler program for the addition of numerals:

This shows, via a translation to numerals,

Interestingly, the SLD-derivation above contains the ‘entangled’ terms and which are neither numerals nor lists, and we believe that such ‘dual’ syntactic objects—which under the conventional doctrine of programming are not well-typed—are characteristic for analogical reasoning across different domains and deserve special attention (this is discussed in sec:Discussion).

The programs in Example exa:Plus are syntactically almost identical. We now give an example in which the programs differ more substantially.

Example 5

In [18] the authors derive a mapping between the program from Example exa:Plus above and the program

which computes list membership. We can now ask—by analogy to [18]—whether can be simulated by (and vice versa) according to our definition. Define the transfer programs

and

It is not hard to compute

which shows that membership can indeed be simulated by the program for appending lists, i.e.,

(4)

Interestingly enough, the transfer programs , which in combination permute the first two arguments of and ‘forget’ about the last one, resemble the mapping computed in [18] by other means. The converse of (equ:Mem_) fails, roughly, since does not contain enough ‘syntactic structure’ to simulate . The intuitive reason is that is a binary predicate, whereas is ternary.

The following simple example shows that analogy and (logical) equivalence are ‘orthogonal’ concepts.

Example 6

The empty program is equivalent to the propositional program consisting of a single rule. Since we cannot obtain the rule from the empty program via composition, equivalence does not imply analogy. For the other direction, a simple computation shows that the programs and are analogous; however, and are not SLD-equivalent.

5 Logic Program Forms

In this section, we study the process of deriving abstract forms of concrete programs from a ‘known’ source domain which can then be instantiated in a ‘novel’ target domain to obtain analogous programs.

We are now ready to introduce the main notion of the paper.

Definition 4

A (logic program) form is any well-formed expression containing programs, program variables, and the composition operations introduced above. Formally, a form is defined by the grammar

where is a substitution, is a symbol, is a program, and is a program variable.

We will usually write forms in boldface to distinguish them from ordinary programs.

Logic program forms induce transformations of programs in the obvious way by replacing program variables with concrete programs. A particularly interesting form, derived from the definition of simulation, is given, for any program , by

Of course, we can now reformulate def:analogy by saying that is simulated by if, and only if, there are some transfer programs such that For instance, we can reformulate the program from Example exa:Plus as

This shows that program forms subsume the concepts of simulation and analogy. Forms are strictly ‘more expressive’ than analogies in the sense that two programs may share some abstract property and still not be analogous according to def:analogy. More precisely, for any form and programs we can say that and share the abstract property expressed by ; however, in general, we cannot expect and to be analogous. This shows that program forms can capture, in a functorial way, abstract similarities which are ‘invisible’ under the lens of analogy.

Example 7

In Example exa:Even we have constructed the program , representing the even numbers, from by inheriting its fact and by iterating its proper rule once. By replacing in (equ:varepsilon) with a program variable , we arrive at the form

(5)

We can now instantiate this form with arbitrary programs to transfer the concept of being ‘even’ to other domains. For example, consider the program

for reversing lists.444Recall from Example exa:Plus that uses the predicate (i.e., symbol) instead of . By instantiating the form (equ:Even) with , we obtain the program given by

One can verify that is a program for reversing lists of even length. Similarly, if is a program for sorting lists, then is a program for sorting ‘even’ lists and so on.

We now come back to Example exa:Plus from the perspective of logic program forms.

Example 8

Reconsider the program of Example exa:Plus given by

We now want to derive a form from which abstractly represents addition. Our first observation is that is, essentially, the concatenation of in the first and last parameter together with a middle part. Formally, we can deconcatenate by

By generalizing the two instances of in the above expression, we immediately obtain the form given by

We can now instantiate this form to obtain analogous programs for addition. For instance, we can derive the program of Example exa:Plus by instantiating the form with the -type program of Example exa:omega_ as follows:

As a further example, we want to define the ‘addition’ of (binary) trees by instantiating the form with the -type program . Note that we now have multiple choices:

  1. As in the case of lists, we can choose to ‘bind’ the variables and .

  2. Additionally, since contains two bound variables (i.e, ), we have four more possibilities.

Let us first consider the program

(6)

given by

This program ‘appends’ the tree in the second argument to each leaf of the symmetric tree in the first argument.

Note that all of the above programs are syntactically almost identical, e.g., we can transform into via a simple rewriting of terms. The next program shows that we can derive programs from which syntactically differ more substantially from the above programs. Concretely, the program

given by

is SLD-equivalent to program (equ:Plus_tau). However, in some situations, this more complicated representation is beneficial. For example, we can now remove the second and third body term to obtain the more compact program

This program, in analogy to program (equ:Plus_tau), ‘appends’ the tree in the second argument to each leaf of the not necessarily symmetric tree in the first argument and thus generalizes (equ:Plus_tau).

The above examples motivate the following Procedure A for the transfer of knowledge from a ‘known’ source domain to a ‘novel’ target domain via logic program forms:

  1. Decompose the source program into factors , .

  2. Replace the programs in the decomposition of by program variables to obtain the form .

  3. Instantiate the form with ‘known’ programs from the target domain to obtain the target program .

  4. If necessary, specialize the program —given some further information on the specification of the intended program—to the concrete domain at hand.

Of course, each step of Procedure A is, in its current form, vague and we leave its formalization as a starting point for future research.

6 Related Work

Arguably, the most prominent (symbolic) model of analogical reasoning to date is Gentner’s Structure-Mapping Theory (or SMT) [6], first implemented in [5]. Our approach shares with Gentner’s SMT its symbolic and syntactic nature. However, while in SMT mappings are constructed with respect to meta-logical considerations—for instance, Gentner’s systematicity principle prefers connected knowledge over independent facts—in our framework mappings are realized as (transfer) programs satisfying mathematically well-defined properties.

Some work has been done on analogical reasoning in the setting of logic programming. Most notably are the early works of [9, 10, 11], which implement Winston’s principle (cf. [19]):

Assume that the premises logically imply in the source domain and that the analogous premises hold in the target domain. Then we can conclude in the target domain which is analogous to .

In practice, the main difficulty for implementing this principle is the detection of two syntactic objects being analogous. In our framework, the analogical mapping between ‘similar’ logical atoms is implicitly established in the computation of transfer programs, thus respecting the syntactic structure of the encapsulating programs.

Finally, we briefly want to mention the works of [4], [3], and [18], dealing with analogical reasoning in logic programming, and the works of [17, 13] which study the nonmonotonic aspects of analogies.

7 Discussion and Future Work

In this paper, we proposed an algebraic model of analogical reasoning in logic programming. More precisely, we introduced algebraic operations for program modularity and defined the notions of analogy and logic program forms in terms of syntactic decompositions of programs.

In general, ‘creative’ analogies contain some form of unexpectedness, in the sense that objects (possibly from different domains) are put together in a novel and often unforeseen way. For instance, in Example exa:Plus we have—directly—used the (seemingly) simpler program for addition of numbers in the list domain for appending lists, where ‘entangled’ terms of the form and , which are neither numerals nor lists, occurred. We believe that ‘dual’ syntactic objects of this form—which under the conventional doctrine of programming are not well-typed (see, for example, the remarks in [1, §5.8.2])—are characteristic for ‘creative’ analogies across different domains and deserve special attention. More precisely, if we think of (logic) programs as ‘tools’ for solving problems, then using those tools ‘creatively’ often requires, as in the example mentioned above, a tight coupling between seemingly incompatible objects. However, if those objects are statically typed by the programmer, then this might prevent the creation and exploitation of useful analogical ‘bridges’ for knowledge transfer.

In a broader sense, this paper is a first step towards an algebraic (and mainly syntactic) theory of analogical reasoning in logic-based knowledge representation and reasoning systems, with potential applications to fundamental AI-problems like computational learning and creativity.

7.1 Outlook

A major task is to formalize each step of Procedure A (cf. sec:Forms):

  1. What are the considerations when systematically computing decompositions of and under which conditions do (unique) ‘prime’ decompositions exist?

  2. Which factors in the decomposition of should be replaced by program variables? (For instance, in Example exa:Plus2, when computing the form we have not replaced the middle factor dealing with the variable by a program variable).

  3. How can we systematically choose the right programs in the target domain to obtain the intended target program ? (For instance, in Example exa:Plus2, it does not make much sense to substitute the program into the form ).

  4. How can we systematically incorporate additional information for the specification of the target program (e.g., in the form of positive and negative examples) into our framework?

For practical concerns, efficient implementations of Procedure A are of interest. From a theoretical point of view, a mathematical study of (the relationship between) modularity, analogy, and logic program forms is desirable.

Analogical proportions are statements of the form ‘ is to as is to ’ and are another form of analogical reasoning. It is interesting to incorporate analogical proportions into our framework by using the algebraic operations introduced in this paper.

Another interesting line of research is to recast the ideas in this paper to nonmonotonic logic programming under the answer set semantics (cf. [2]). This task is non-trivial due to negation as failure occurring in rule bodies (and heads), and we want to mention that some work has already been done in this direction (cf. [17, 13, 14]).

Finally, motivated by the discussion above, we believe that it is interesting to investigate the role of (polymorphic) (un)typing in computational models of analogy-making for computational learning and creativity.

Acknowledgements

This research has been supported by the Austrian Science Fund (FWF) project I 2671-N35.

References

  • [1] Apt, K.R.: From Logic Programming to Prolog. C.A.R. Hoare Series, Prentice Hall, Prentice Hall, Englewood Cliffs, NJ (1997)
  • [2] Brewka, G., Eiter, T., Truszczynski, M.: Answer set programming at a glance. Communications of the ACM 54(12), 92–103 (Dec 2011)
  • [3]

    Costantini, S., Lanzarone, G.A., Sbarbaro, L.: A formal definition and a sound implementation of analogical reasoning in logic programming. Annals of Mathematics and Artificial Intelligence

    14(1), 17–36 (1995)
  • [4] Davies, T.R., Russell, S.J.: A logical approach to reasoning by analogy. In: McDermott, J.P. (ed.) IJCAI 1987, pp. 264–270. Morgan Kaufmann (1987)
  • [5] Falkenhainer, B., Forbus, K.D., Gentner, D.: The structure-mapping engine: algorithm and examples. Artificial Intelligence 41(1), 1–63 (1989)
  • [6] Gentner, D.: Structure-mapping: a theoretical framework for analogy. Cognitive Science 7(2), 155–170 (1983)
  • [7] Gust, H., Krumnack, U., Kühnberger, K.U., Schwering, A.: Analogical reasoning: a core of cognition. Künstliche Intelligenz 22(1), 8–12 (2008)
  • [8] Hall, R.P.: Computational approaches to analogical reasoning: a comparative analysis. Artificial Intelligence 39(1), 39–120 (1989)
  • [9] Haraguchi, M.: Analogical reasoning using transformations of rules. In: Wada, E. (ed.) Logic Programming ’85, pp. 56–65. LNCS 221, Springer-Verlag, Berlin/Heidelberg (1986)
  • [10] Haraguchi, M., Arikawa, S.: A foundation of reasoning by analogy: analogical union of logic programs. In: Wada, E. (ed.) LP 1986, pp. 58–69. Springer-Verlag, Berlin/Heidelberg (1986)
  • [11] Haraguchi, M., Arikawa, S.: Reasoning by analogy as a partial identity between models. In: Jantke, K.P. (ed.) Analogical and Inductive Inference, pp. 61–87. LNCS 265, Springer-Verlag, Berlin/Heidelberg (1987)
  • [12] Hofstadter, D.: Analogy as the core of cognition. In: Gentner, D., Holyoak, K.J., Kokinov, B.K. (eds.) The Analogical Mind: Perspectives from Cognitive Science, pp. 499–538. MIT Press/Bradford Book, Cambridge MA (2001)
  • [13] Iwayama, N., Satoh, K., Arima, J.: A formalization of generalization-based analogy in general logic programs. In: Neumann, B. (ed.) EOAI 1992, pp. 129–133. John Wiley & Sons, Chichester (1992)
  • [14] Prade, H., Richard, G.: Cataloguing/analogizing: a nonmonotonic view. International Journal of Intelligent Systems 26(12), 1176–1195 (2011)
  • [15] Prade, H., Richard, G. (eds.): Computational Approaches to Analogical Reasoning: Current Trends. Studies in Computational Intelligence, Springer-Verlag, Berlin/Heidelberg (2014)
  • [16] Prade, H., Richard, G.: A short introduction to computational trends in analogical reasoning. In: Prade, H., Richard, G. (eds.) Approaches to Analogical Reasoning: Current Trends, pp. 1–22. Studies in Computational Intelligence 548, Springer-Verlag, Berlin/Heidelberg (2014)
  • [17] Qiu, G.: Nonmonotonic logic for analogical reasoning. In: Greiner, R., Subramanian, D. (eds.) Working Notes of the 1994 AAAI Fall Symposium on Relevance. vol. AAAI Tech. Rep. 94–02, pp. 161–164. New Orleans, Louisiana (1994)
  • [18] Tausend, B., Bell, S.: Analogical reasoning for logic programming. In: Kodratoff, Y. (ed.) EWSL 1991, pp. 391–397. LNCS 482, Springer-Verlag, Berlin/Heidelberg (1991)
  • [19] Winston, P.H.: Learning new principles from precedents and exercises. Artificial Intelligence 19(3), 321–350 (1982)