DeepAI AI Chat
Log In Sign Up

FaiRR: Faithful and Robust Deductive Reasoning over Natural Language

03/19/2022
by   Soumya Sanyal, et al.
University of Southern California
9

Transformers have been shown to be able to perform deductive reasoning on a logical rulebase containing rules and statements written in natural language. Recent works show that such models can also produce the reasoning steps (i.e., the proof graph) that emulate the model's logical reasoning process. Currently, these black-box models generate both the proof graph and intermediate inferences within the same model and thus may be unfaithful. In this work, we frame the deductive logical reasoning task by defining three modular components: rule selection, fact selection, and knowledge composition. The rule and fact selection steps select the candidate rule and facts to be used and then the knowledge composition combines them to generate new inferences. This ensures model faithfulness by assured causal relation from the proof step to the inference reasoning. To test our framework, we propose FaiRR (Faithful and Robust Reasoner) where the above three components are independently modeled by transformers. We observe that FaiRR is robust to novel language perturbations, and is faster at inference than previous works on existing reasoning datasets. Additionally, in contrast to black-box generative models, the errors made by FaiRR are more interpretable due to the modular approach.

READ FULL TEXT
05/25/2022

RobustLR: Evaluating Robustness to Logical Perturbation in Deductive Reasoning

Transformers have been shown to be able to perform deductive reasoning o...
04/06/2020

Multi-Step Inference for Reasoning Over Paragraphs

Complex reasoning over text requires understanding and chaining together...
11/06/2020

Extending Equational Monadic Reasoning with Monad Transformers

There is a recent interest for the verification of monadic programs usin...
12/24/2020

ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language

Transformers have been shown to emulate logical deduction over natural l...
04/06/2023

Handling Wikidata Qualifiers in Reasoning

Wikidata is a knowledge graph increasingly adopted by many communities f...
09/16/2022

Dynamic Generation of Interpretable Inference Rules in a Neuro-Symbolic Expert System

We present an approach for systematic reasoning that produces human inte...
09/30/2020

Measuring Systematic Generalization in Neural Proof Generation with Transformers

We are interested in understanding how well Transformer language models ...