Generating Natural Language Proofs with Verifier-Guided Search

05/25/2022
by   Kaiyu Yang, et al.
0

Deductive reasoning (drawing conclusions from assumptions) is a challenging problem in NLP. In this work, we focus on proof generation: given a hypothesis and a set of supporting facts in natural language, the model generates a proof tree indicating how to deduce the hypothesis from supporting facts. Instead of generating the entire proof in one shot, prior work has demonstrated the promise of stepwise generation but achieved limited success on real-world data. Existing stepwise methods struggle to generate proof steps that are both valid and relevant. In this paper, we present a novel stepwise method NLProofS (Natural Language Proof Search), which learns to generate relevant steps conditioning on the hypothesis. At the core of our approach, we train an independent verifier to check the validity of proof steps. Instead of generating steps greedily, we search for proofs maximizing a global proof score judged by the verifier. NLProofS achieves state-of-the-art performance on EntailmentBank and RuleTaker. For example, it improves the percentage of correctly predicted proofs from 20.9 EntailmentBank. This is the first time stepwise methods have led to better generation of challenging human-authored proofs.

READ FULL TEXT
06/02/2021

multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning

We focus on a type of linguistic formal reasoning where the goal is to r...
12/11/2017

Coqatoo: Generating Natural Language Versions of Coq Proofs

Due to their numerous advantages, formal proofs and proof assistants, su...
12/24/2020

ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language

Transformers have been shown to emulate logical deduction over natural l...
01/16/2022

Natural Language Deduction through Search over Statement Compositions

In settings from fact-checking to question answering, we frequently want...
07/06/2021

Probabilistic Graph Reasoning for Natural Proof Generation

In this paper, we investigate the problem of reasoning over natural lang...
12/05/2017

Determinism in the Certification of UNSAT Proofs

The search for increased trustworthiness of SAT solvers is very active a...
07/28/2021

Goal-Oriented Script Construction

The knowledge of scripts, common chains of events in stereotypical scena...

Code Repositories

NLProofS

Code for the paper "Generating Natural Language Proofs with Verifier-Guided Search": https://arxiv.org/abs/2205.12443


view repo