Generating Natural Language Proofs with Verifier-Guided Search

by   Kaiyu Yang, et al.

Deductive reasoning (drawing conclusions from assumptions) is a challenging problem in NLP. In this work, we focus on proof generation: given a hypothesis and a set of supporting facts in natural language, the model generates a proof tree indicating how to deduce the hypothesis from supporting facts. Instead of generating the entire proof in one shot, prior work has demonstrated the promise of stepwise generation but achieved limited success on real-world data. Existing stepwise methods struggle to generate proof steps that are both valid and relevant. In this paper, we present a novel stepwise method NLProofS (Natural Language Proof Search), which learns to generate relevant steps conditioning on the hypothesis. At the core of our approach, we train an independent verifier to check the validity of proof steps. Instead of generating steps greedily, we search for proofs maximizing a global proof score judged by the verifier. NLProofS achieves state-of-the-art performance on EntailmentBank and RuleTaker. For example, it improves the percentage of correctly predicted proofs from 20.9 EntailmentBank. This is the first time stepwise methods have led to better generation of challenging human-authored proofs.


multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning

We focus on a type of linguistic formal reasoning where the goal is to r...

Coqatoo: Generating Natural Language Versions of Coq Proofs

Due to their numerous advantages, formal proofs and proof assistants, su...

ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language

Transformers have been shown to emulate logical deduction over natural l...

Natural Language Deduction through Search over Statement Compositions

In settings from fact-checking to question answering, we frequently want...

Probabilistic Graph Reasoning for Natural Proof Generation

In this paper, we investigate the problem of reasoning over natural lang...

Determinism in the Certification of UNSAT Proofs

The search for increased trustworthiness of SAT solvers is very active a...

Goal-Oriented Script Construction

The knowledge of scripts, common chains of events in stereotypical scena...

Code Repositories


Code for the paper "Generating Natural Language Proofs with Verifier-Guided Search":

view repo