Nonparametric Hamiltonian Monte Carlo

06/18/2021
by   Carol Mak, et al.
0

Probabilistic programming uses programs to express generative models whose posterior probability is then computed by built-in inference engines. A challenging goal is to develop general purpose inference algorithms that work out-of-the-box for arbitrary programs in a universal probabilistic programming language (PPL). The densities defined by such programs, which may use stochastic branching and recursion, are (in general) nonparametric, in the sense that they correspond to models on an infinite-dimensional parameter space. However standard inference algorithms, such as the Hamiltonian Monte Carlo (HMC) algorithm, target distributions with a fixed number of parameters. This paper introduces the Nonparametric Hamiltonian Monte Carlo (NP-HMC) algorithm which generalises HMC to nonparametric models. Inputs to NP-HMC are a new class of measurable functions called "tree representable", which serve as a language-independent representation of the density functions of probabilistic programs in a universal PPL. We provide a correctness proof of NP-HMC, and empirically demonstrate significant performance improvements over existing approaches on several nonparametric examples.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

04/07/2018

Discontinuous Hamiltonian Monte Carlo for Probabilistic Programs

Hamiltonian Monte Carlo (HMC) is the dominant statistical inference algo...
10/29/2019

Divide, Conquer, and Combine: a New Inference Strategy for Probabilistic Programs with Stochastic Support

Universal probabilistic programming systems (PPSs) provide a powerful an...
03/06/2019

LF-PPL: A Low-Level First Order Probabilistic Programming Language for Non-Differentiable Models

We develop a new Low-level, First-order Probabilistic Programming Langua...
02/04/2021

HMC, an Algorithms in Data Mining, the Functional Analysis approach

The main purpose of this paper is to facilitate the communication betwee...
04/14/2021

Measuring diachronic sense change: new models and Monte Carlo methods for Bayesian inference

In a bag-of-words model, the senses of a word with multiple meanings, e....
03/16/2018

Nesting Probabilistic Programs

We formalize the notion of nesting probabilistic programming queries and...
06/13/2012

Church: a language for generative models

We introduce Church, a universal language for describing stochastic gene...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.