Venture: a higher-order probabilistic programming platform with programmable inference

04/01/2014
by   Vikash Mansinghka, et al.
0

We describe Venture, an interactive virtual machine for probabilistic programming that aims to be sufficiently expressive, extensible, and efficient for general-purpose use. Like Church, probabilistic models and inference problems in Venture are specified via a Turing-complete, higher-order probabilistic language descended from Lisp. Unlike Church, Venture also provides a compositional language for custom inference strategies built out of scalable exact and approximate techniques. We also describe four key aspects of Venture's implementation that build on ideas from probabilistic graphical models. First, we describe the stochastic procedure interface (SPI) that specifies and encapsulates primitive random variables. The SPI supports custom control flow, higher-order probabilistic procedures, partially exchangeable sequences and "likelihood-free" stochastic simulators. It also supports external models that do inference over latent variables hidden from Venture. Second, we describe probabilistic execution traces (PETs), which represent execution histories of Venture programs. PETs capture conditional dependencies, existential dependencies and exchangeable coupling. Third, we describe partitions of execution histories called scaffolds that factor global inference problems into coherent sub-problems. Finally, we describe a family of stochastic regeneration algorithms for efficiently modifying PET fragments contained within scaffolds. Stochastic regeneration linear runtime scaling in cases where many previous approaches scaled quadratically. We show how to use stochastic regeneration and the SPI to implement general-purpose inference strategies such as Metropolis-Hastings, Gibbs sampling, and blocked proposals based on particle Markov chain Monte Carlo and mean-field variational inference techniques.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/03/2015

A New Approach to Probabilistic Programming Inference

We introduce and demonstrate a new approach to inference in expressive p...
research
07/09/2014

Learning Probabilistic Programs

We develop a technique for generalising from data in which models are sa...
research
10/23/2020

Accelerating Metropolis-Hastings with Lightweight Inference Compilation

In order to construct accurate proposers for Metropolis-Hastings Markov ...
research
12/14/2016

Encapsulating models and approximate inference programs in probabilistic modules

This paper introduces the probabilistic module interface, which allows e...
research
11/06/2014

Sublinear-Time Approximate MCMC Transitions for Probabilistic Programs

Probabilistic programming languages can simplify the development of mach...
research
07/21/2022

Language Model Cascades

Prompted models have demonstrated impressive few-shot learning abilities...
research
10/29/2019

Divide, Conquer, and Combine: a New Inference Strategy for Probabilistic Programs with Stochastic Support

Universal probabilistic programming systems (PPSs) provide a powerful an...

Please sign up or login with your details

Forgot password? Click here to reset