Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions

10/15/2019
by   Lars Buesing, et al.
0

A plethora of problems in AI, engineering and the sciences are naturally formalized as inference in discrete probabilistic models. Exact inference is often prohibitively expensive, as it may require evaluating the (unnormalized) target density on its entire domain. Here we consider the setting where only a limited budget of calls to the unnormalized density oracle is available, raising the challenge of where in the domain to allocate these function calls in order to construct a good approximate solution. We formulate this problem as an instance of sequential decision-making under uncertainty and leverage methods from reinforcement learning for probabilistic inference with budget constraints. In particular, we propose the TreeSample algorithm, an adaptation of Monte Carlo Tree Search to approximate inference. This algorithm caches all previous queries to the density oracle in an explicit search tree, and dynamically allocates new queries based on a "best-first" heuristic for exploration, using existing upper confidence bound methods. Our non-parametric inference method can be effectively combined with neural networks that compile approximate conditionals of the target, which are then used to guide the inference search and enable generalization across multiple target distributions. We show empirically that TreeSample outperforms standard approximate inference methods on synthetic factor graphs.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/20/2020

Monte-Carlo Graph Search for AlphaZero

The AlphaZero algorithm has been successfully applied in a range of disc...
research
12/06/2022

Proposal of a Score Based Approach to Sampling Using Monte Carlo Estimation of Score and Oracle Access to Target Density

Score based approaches to sampling have shown much success as a generati...
research
03/12/2019

Elements of Sequential Monte Carlo

A core problem in statistics and probabilistic machine learning is to co...
research
02/14/2022

Anytime Capacity Expansion in Medical Residency Match by Monte Carlo Tree Search

This paper considers the capacity expansion problem in two-sided matchin...
research
09/08/2017

Roll-back Hamiltonian Monte Carlo

We propose a new framework for Hamiltonian Monte Carlo (HMC) on truncate...
research
09/12/2023

Update Monte Carlo tree search (UMCTS) algorithm for heuristic global search of sizing optimization problems for truss structures

Sizing optimization of truss structures is a complex computational probl...
research
02/24/2014

Variational Particle Approximations

Approximate inference in high-dimensional, discrete probabilistic models...

Please sign up or login with your details

Forgot password? Click here to reset