Language Model Cascades

07/21/2022
by   David Dohan, et al.
5

Prompted models have demonstrated impressive few-shot learning abilities. Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. These compositions are probabilistic models, and may be expressed in the language of graphical models with random variables whose values are complex data types such as strings. Cases with control flow and dynamic structure require techniques from probabilistic programming, which allow implementing disparate model structures and inference strategies in a unified language. We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use. We refer to the resulting programs as language model cascades.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/11/2015

Lazy Factored Inference for Functional Probabilistic Programming

Probabilistic programming provides the means to represent and reason abo...
research
10/19/2021

flip-hoisting: Exploiting Repeated Parameters in Discrete Probabilistic Programs

Probabilistic programming is emerging as a popular and effective means o...
research
10/02/2018

Automated learning with a probabilistic programming language: Birch

This work offers a broad perspective on probabilistic modeling and infer...
research
09/16/2022

Psychologically-informed chain-of-thought prompts for metaphor understanding in large language models

Probabilistic models of language understanding are interpretable and str...
research
04/01/2014

Venture: a higher-order probabilistic programming platform with programmable inference

We describe Venture, an interactive virtual machine for probabilistic pr...
research
07/29/2020

Connecting actuarial judgment to probabilistic learning techniques with graph theory

Graphical models have been widely used in applications ranging from medi...

Please sign up or login with your details

Forgot password? Click here to reset