Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP

12/28/2022
by   Omar Khattab, et al.
0

Retrieval-augmented in-context learning has emerged as a powerful approach for addressing knowledge-intensive tasks using frozen language models (LM) and retrieval models (RM). Existing work has combined these in simple "retrieve-then-read" pipelines in which the RM retrieves passages that are inserted into the LM prompt. To begin to fully realize the potential of frozen LMs and RMs, we propose Demonstrate-Search-Predict (DSP), a framework that relies on passing natural language texts in sophisticated pipelines between an LM and an RM. DSP can express high-level programs that bootstrap pipeline-aware demonstrations, search for relevant passages, and generate grounded predictions, systematically breaking down problems into small transformations that the LM and RM can handle more reliably. We have written novel DSP programs for answering questions in open-domain, multi-hop, and conversational settings, establishing in early evaluations new state-of-the-art in-context learning results and delivering 37-120 vanilla LM (GPT-3.5), a standard retrieve-then-read pipeline, and a contemporaneous self-ask pipeline, respectively. We release DSP at https://github.com/stanfordnlp/dsp

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/23/2023

Query Rewriting for Retrieval-Augmented Large Language Models

Large Language Models (LLMs) play a powerful Reader of the Retrieve-then...
research
09/21/2022

Generate rather than Retrieve: Large Language Models are Strong Context Generators

Knowledge-intensive tasks, such as open-domain question answering (QA), ...
research
04/22/2022

Autoregressive Search Engines: Generating Substrings as Document Identifiers

Knowledge-intensive language tasks require NLP systems to both provide t...
research
05/11/2023

Active Retrieval Augmented Generation

Despite the remarkable ability of large language models (LMs) to compreh...
research
08/21/2023

RaLLe: A Framework for Developing and Evaluating Retrieval-Augmented Large Language Models

Retrieval-augmented large language models (R-LLMs) combine pre-trained l...
research
05/23/2023

Improving Language Models via Plug-and-Play Retrieval Feedback

Large language models (LLMs) exhibit remarkable performance across vario...

Please sign up or login with your details

Forgot password? Click here to reset