Flexible Operations for Natural Language Deduction

by   Kaj Bostrom, et al.

An interpretable system for complex, open-domain reasoning needs an interpretable meaning representation. Natural language is an excellent candidate – it is both extremely expressive and easy for humans to understand. However, manipulating natural language statements in logically consistent ways is hard. Models have to be precise, yet robust enough to handle variation in how information is expressed. In this paper, we describe ParaPattern, a method for building models to generate logical transformations of diverse natural language inputs without direct human supervision. We use a BART-based model (Lewis et al., 2020) to generate the result of applying a particular logical operation to one or more premise statements. Crucially, we have a largely automated pipeline for scraping and constructing suitable training examples from Wikipedia, which are then paraphrased to give our models the ability to handle lexical variation. We evaluate our models using targeted contrast sets as well as out-of-domain sentence compositions from the QASC dataset (Khot et al., 2020). Our results demonstrate that our operation models are both accurate and flexible.


page 1

page 2

page 3

page 4


RobustLR: Evaluating Robustness to Logical Perturbation in Deductive Reasoning

Transformers have been shown to be able to perform deductive reasoning o...

SeqGenSQL – A Robust Sequence Generation Model for Structured Query Language

We explore using T5 (Raffel et al. (2019)) to directly translate natural...

ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language

Transformers have been shown to emulate logical deduction over natural l...

NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language

Rule-based models are attractive for various tasks because they inherent...

Visual Reasoning with Natural Language

Natural language provides a widely accessible and expressive interface f...

Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack

The detection of offensive language in the context of a dialogue has bec...

On modelling the emergence of logical thinking

Recent progress in machine learning techniques have revived interest in ...

Code Repositories


Code for the paper "Flexible Operations for Natural Language Deduction"

view repo