Coqatoo: Generating Natural Language Versions of Coq Proofs

12/11/2017
by   Andrew Bedford, et al.
Université Laval
0

Due to their numerous advantages, formal proofs and proof assistants, such as Coq, are becoming increasingly popular. However, one disadvantage of using proof assistants is that the resulting proofs can sometimes be hard to read and understand, particularly for less-experienced users. To address this issue, we have implemented a tool capable of generating natural language versions of Coq proofs called Coqatoo, which we present in this paper.

READ FULL TEXT VIEW PDF

page 1

page 2

12/05/2017

Analyzing Individual Proofs as the Basis of Interoperability between Proof Systems

We describe the first results of a project of analyzing in which theorie...
12/24/2020

ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language

Transformers have been shown to emulate logical deduction over natural l...
05/25/2022

Generating Natural Language Proofs with Verifier-Guided Search

Deductive reasoning (drawing conclusions from assumptions) is a challeng...
06/02/2021

multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning

We focus on a type of linguistic formal reasoning where the goal is to r...
01/15/2021

The Coq Proof Script Visualiser (coq-psv)

In this work, we present a visualisation tool that is able to process Co...
08/26/2019

Converting ALC Connection Proofs into ALC Sequents

The connection method has earned good reputation in the area of automate...
04/02/2021

An Evaluation of the Archive of Formal Proofs

The Archive of Formal Proofs (AFP) is an online repository of formal pro...

1. Introduction

Due to their numerous advantages, formal proofs and proof assistants, such as Coq, are becoming increasingly popular. However, one disadvantage of using proof assistants is that the resulting proofs can sometimes be hard to read and understand, particularly for less-experienced users. In an attempt to address this issue, Coscoy et al. (Coscoy et al., 1995) developed in 1995 an algorithm capable of generating natural language proofs from Coq proof-terms (i.e., calculus of inductive construction -terms) and implemented their approach in two development environments: CtCoq (et al., 1997; Bertot, 1999) and its successor Pcoq (et al., 2003; Amerkad et al., 2001). Unfortunately, these development environments are no longer available or maintained; Pcoq’s last version dates from 2003 and requires Coq 7.4.

In order to bring this useful feature to modern development environments, we have implemented our own rewriting algorithm: Coqatoo.

2. Overview of Coqatoo

Much like Nuprl’s text generation algorithm 

(Holland-Minkley et al., 1999), Coqatoo generates natural language proofs from high-level proof scripts instead of the low-level proof-terms used by Coscoy et al. By doing so, we can avoid the verbosity that comes from using low-level proof-terms (Coscoy, 1996) and avoid losing valuable information such as the tactics that are used, the user’s comments and the variable names.

Coqatoo’s rewriting algorithm can be decomposed in three steps: information extraction, proof tree construction and tactic-based rewriting.

Step 1: Information extraction

Using an instance of the coqtop process and the proof script given as input, Coqatoo executes the tactics one by one and captures the intermediary proof states.

For example, Listing 1 represents the initial state of Listing 3’s proof and Listing 2 represents the state after executing the first intros tactic.

  1 subgoal
  ============================
  forall P Q R : Prop, (P /\ Q -> R) <-> (P -> Q -> R)
Listing 1: State before executing the first intros tactic
  1 subgoal
  P, Q, R : Prop
  ============================
  (P /\ Q -> R) <-> (P -> Q -> R)
Listing 2: State after executing the first intros tactic

These intermediary states, which contain the current assumptions and remaining goals, allow us to identify the changes caused by a tactic’s execution (e.g., added/removed variables, hypotheses or subgoals).

Step 2: Proof tree construction

We then build a tree representating the proof’s structure (e.g., Figure 1). This is a necessary step for our rewriting algorithm as it allows it to determine where bullets should be inserted and when lines should be indented.

Figure 1. Proof tree of Listing 3

Step 3: Tactic-based rewriting

Finally, we generate the actual final natural language version of the proof using simple rewriting rules. Each supported tactic has its own set of rules. For example, for the intros tactic we first determine the types of the objects that are introduced. If they are variables, then we produce a sentence of the form "Assume that ... are arbitrary objects of type ...". If they are hypotheses, then we instead produce a sentence of the form "Suppose that ... are true". Finally, we insert a sentence indicating what is left to prove: "Let us show that ...".

Note that the sentences that we use to produce natural language versions are kept in files that are separate from the code. This allows Coqatoo to support multiple languages and proof styles. For the moment, it can output proofs in English or French, in plain text or in annotation mode (see Listing 

4 for example). In annotation mode, each tactic is accompanied with an informal explanation. We believe that this format will be particularly useful for new Coq users.

2.1. Example

To illustrate our approach, consider the proof script in Listing 3 and Coqatoo’s output in Listing 4.

  Lemma conj_imp_equiv : forall P Q R:Prop,
    ((P /\ Q -> R) <-> (P -> Q -> R)).
  Proof.
    intros. split. intros H HP HQ. apply H. apply conj. assumption. assumption.
    intros H HPQ. inversion HPQ. apply H. assumption. assumption.
  Qed.
Listing 3: Proof script given as input
  Lemma conj_imp_equiv : forall P Q R:Prop, ((P /\ Q -> R) <-> (P -> Q -> R)).
  Proof.
  (* Assume that P, Q and R are arbitrary objects of type Prop. Let us show that (P /\ Q -> R) <-> (P -> Q -> R) is true. *) intros.
  split.
    - (* Case (P /\ Q -> R) -> P -> Q -> R: *)
      (* Suppose that P, Q and P /\ Q -> R are true. Let us show that R is true. *) intros H HP HQ.
      (* By our hypothesis P /\ Q -> R, we know that R is true if P /\ Q  is true. *) apply H.
      apply conj.
      -- (* Case P: *)
         (* True, because it is one of our assumptions. *) assumption.
      -- (* Case Q: *)
         (* True, because it is one of our assumptions. *) assumption.
    - (* Case (P -> Q -> R) -> P /\ Q -> R: *)
      (* Suppose that P /\ Q and P -> Q -> R are true. Let us show that R is true. *) intros H HPQ.
      (* By inversion on P /\ Q, we know that P, Q are also true. *) inversion HPQ.
      (* By our hypothesis P -> Q -> R, we know that R is true if P and Q are true. *) apply H.
      -- (* Case P: *)
         (* True, because it is one of our assumptions. *) assumption.
      -- (* Case Q: *)
         (* True, because it is one of our assumptions. *) assumption.
  Qed.
Listing 4: Output in annotation mode

2.2. Comparison

Compared to Coscoy et al., our approach presents a few disadvantages and advantages.

Disadvantages

  • It only works on proofs whose tactics are supported (see Section 3), while the approach of Coscoy et al. worked on any proof.

  • It may require additional verifications to ensure that unecessary information (e.g., an assertion which isn’t used) is not included in the generated proof.

Advantages

  • It enables us to more easily control the size and verbosity of the generated proof (one or two sentences per tactic by default).

  • It maintains the order and structure of the user’s original proof script; this is not necessarily the case in Coscoy et al.

3. Future Work

Coqatoo is only a proof-of-concept for the moment. As such, there remains much to be done before it can be of real use.

Increase the number of supported tactics

The number of tactics that it supports is limited to only a handful (see Coqatoo’s GitHub repository (Bedford, 2017) for more details). We expect that, with the help of the community, we will be able to support enough tactics to generate natural language versions of most proofs in Software Foundations (Pierce et al., 2010).

Add partial support for automation

In regards to automation, Coqatoo only supports the auto tactic: if the auto tactic is present within the script, it is replaced with info_auto in order to obtain the sequence of tactics that is used by auto. We plan on adding partial support for automation in the future, starting with the chaining operator ";". To support this operator we will use our tree representation of proofs to "distribute" tactics on branches.

Integration with development environments

Once it is sufficiently developed, we plan on integrating our utility in modern Coq development environments such as CoqIDE and ProofGeneral.

Add a LaTeX output mode

We plan on adding a LaTeX output mode so that the generated proofs can be easily inserted into LaTeX documents.

Acknowledgements.
We would like to thank Josée Desharnais, Nadia Tawbi, Souad El Hatib and the reviewers for their comments. We would also like to thank the Coq community for the large number of resources and tutorials that are available online.

References

  • (1)
  • Amerkad et al. (2001) Ahmed Amerkad, Yves Bertot, Loïc Pottier, and Laurence Rideau. 2001. Mathematics and proof presentation in Pcoq. Ph.D. Dissertation. INRIA.
  • Bedford (2017) Andrew Bedford. 2017. Coqatoo’s Repository. https://github.com/andrew-bedford/coqatoo. (2017).
  • Bertot (1999) Yves Bertot. 1999. The CtCoq system: Design and architecture. Formal aspects of Computing 11, 3 (1999), 225–243.
  • Coscoy (1996) Yann Coscoy. 1996. A Natural Language Explanation for Formal Proofs. In Logical Aspects of Computational Linguistics, First International Conference, LACL ’96, Nancy, France, September 23-25, 1996, Selected Papers. 149–167. https://doi.org/10.1007/BFb0052156
  • Coscoy et al. (1995) Yann Coscoy, Gilles Kahn, and Laurent Théry. 1995. Extracting Text from Proofs. In Typed Lambda Calculi and Applications, Second International Conference on Typed Lambda Calculi and Applications, TLCA ’95, Edinburgh, UK, April 10-12, 1995, Proceedings. 109–123. https://doi.org/10.1007/BFb0014048
  • et al. (1997) Yves Bertot et al. 1997. CtCoq. https://www-sop.inria.fr/croap/ctcoq/ctcoq-eng.html. (1997).
  • et al. (2003) Yves Bertot et al. 2003. Pcoq. http://www-sop.inria.fr/lemme/pcoq/. (2003).
  • Holland-Minkley et al. (1999) Amanda M. Holland-Minkley, Regina Barzilay, and Robert L. Constable. 1999. Verbalization of High-Level Formal Proofs. In

    Proceedings of the Sixteenth National Conference on Artificial Intelligence and Eleventh Conference on Innovative Applications of Artificial Intelligence, July 18-22, 1999, Orlando, Florida, USA.

    277–284.
    http://www.aaai.org/Library/AAAI/1999/aaai99-041.php
  • Pierce et al. (2010) Benjamin C Pierce, Chris Casinghino, Marco Gaboardi, Michael Greenberg, Cătălin Hriţcu, Vilhelm Sjöberg, and Brent Yorgey. 2010. Software foundations. http://www.cis.upenn.edu/bcpierce/sf/current/index.html.