DeepAI AI Chat
Log In Sign Up

Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2

by   Gregor Betz, et al.

Thinking aloud is an effective meta-cognitive strategy human reasoners apply to solve difficult problems. We suggest to improve the reasoning ability of pre-trained neural language models in a similar way, namely by expanding a task's context with problem elaborations that are dynamically generated by the language model itself. Our main result is that dynamic problem elaboration significantly improves the zero-shot performance of GPT-2 in a deductive reasoning and natural language inference task: While the model uses a syntactic heuristic for predicting an answer, it is capable (to some degree) of generating reasoned additional context which facilitates the successful application of its heuristic. We explore different ways of generating elaborations, including fewshot learning, and find that their relative performance varies with the specific problem characteristics (such as problem difficulty). Moreover, the effectiveness of an elaboration can be explained in terms of the degree to which the elaboration semantically coheres with the corresponding problem. In particular, elaborations that are most faithful to the original problem description may boost accuracy by up to 24


page 5

page 8

page 14


Enhance Reasoning Ability of Visual-Language Models via Large Language Models

Pre-trained visual language models (VLM) have shown excellent performanc...

Emergent Analogical Reasoning in Large Language Models

The recent advent of large language models - large neural networks train...

Towards Zero-shot Commonsense Reasoning with Self-supervised Refinement of Language Models

Can we get existing language models and refine them for zero-shot common...

Going Beyond Nouns With Vision Language Models Using Synthetic Data

Large-scale pre-trained Vision Language (VL) models have shown remar...

An Application of Pseudo-Log-Likelihoods to Natural Language Scoring

Language models built using semi-supervised machine learning on large co...

StructGPT: A General Framework for Large Language Model to Reason over Structured Data

In this paper, we study how to improve the zero-shot reasoning ability o...

Probabilistic Graph Reasoning for Natural Proof Generation

In this paper, we investigate the problem of reasoning over natural lang...