Log In Sign Up

Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning

by   Lianhui Qin, et al.

Abductive and counterfactual reasoning, core abilities of everyday human cognition, require reasoning about what might have happened at time t, while conditioning on multiple contexts from the relative past and future. However, simultaneous incorporation of past and future contexts using generative language models (LMs) can be challenging, as they are trained either to condition only on the past context or to perform narrowly scoped text-infilling. In this paper, we propose DeLorean, a new unsupervised decoding algorithm that can flexibly incorporate both the past and future contexts using only off-the-shelf, left-to-right language models and no supervision. The key intuition of our algorithm is incorporating the future through back-propagation, during which, we only update the internal representation of the output while fixing the model parameters. By alternating between forward and backward propagation, DeLorean can decode the output representation that reflects both the left and right contexts. We demonstrate that our approach is general and applicable to two nonmonotonic reasoning tasks: abductive text generation and counterfactual story revision, where DeLorean outperforms a range of unsupervised and some supervised methods, based on automatic and human evaluation.


Reflective Decoding: Unsupervised Paraphrasing and Abductive Reasoning

Pretrained Language Models (LMs) generate text with remarkable quality, ...

Counterfactual Story Reasoning and Generation

Counterfactual reasoning requires predicting how alternative events, con...

Possible Stories: Evaluating Situated Commonsense Reasoning under Multiple Possible Scenarios

The possible consequences for the same context may vary depending on the...

Abductive Commonsense Reasoning

Abductive reasoning is inference to the most plausible explanation. For ...

CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Models

We introduce the CRASS (counterfactual reasoning assessment) data set an...

Unsupervised Editing for Counterfactual Stories

Creating what-if stories requires reasoning about prior statements and p...