Self-Refine: Iterative Refinement with Self-Feedback

by   Aman Madaan, et al.

Like people, LLMs do not always generate the best text for a given generation problem on their first try (e.g., summaries, answers, explanations). Just as people then refine their text, we introduce SELF-REFINE, a framework for similarly improving initial outputs from LLMs through iterative feedback and refinement. The main idea is to generate an output using an LLM, then allow the same model to provide multi-aspect feedback for its own output; finally, the same model refines its previously generated output given its own feedback. Unlike earlier work, our iterative refinement framework does not require supervised training data or reinforcement learning, and works with a single LLM. We experiment with 7 diverse tasks, ranging from review rewriting to math reasoning, demonstrating that our approach outperforms direct generation. In all tasks, outputs generated with SELF-REFINE are preferred by humans and by automated metrics over those generated directly with GPT-3.5 and GPT-4, improving on average by absolute 20


page 28

page 29

page 36

page 37

page 38

page 39

page 40

page 42


Training Language Models with Language Feedback at Scale

Pretrained language models often generate outputs that are not in line w...

SummIt: Iterative Text Summarization via ChatGPT

Existing text summarization systems have made significant progress in re...

FAST: Improving Controllability for Text Generation with Feedback Aware Self-Training

Controllable text generation systems often leverage control codes to dir...

Divide, Evaluate, and Refine: Evaluating and Improving Text-to-Image Alignment with Iterative VQA Feedback

The field of text-conditioned image generation has made unparalleled pro...

Align-Refine: Non-Autoregressive Speech Recognition via Iterative Realignment

Non-autoregressive models greatly improve decoding speed over typical se...

ReDecode Framework for Iterative Improvement in Paraphrase Generation

Generating paraphrases, that is, different variations of a sentence conv...

Refining the Responses of LLMs by Themselves

In this paper, we propose a simple yet efficient approach based on promp...

Please sign up or login with your details

Forgot password? Click here to reset