Gaussian Process Bandit Optimization of the Thermodynamic Variational Objective

10/29/2020
by   Vu Nguyen, et al.
11

Achieving the full promise of the Thermodynamic Variational Objective (TVO), a recently proposed variational lower bound on the log evidence involving a one-dimensional Riemann integral approximation, requires choosing a "schedule" of sorted discretization points. This paper introduces a bespoke Gaussian process bandit optimization method for automatically choosing these points. Our approach not only automates their one-time selection, but also dynamically adapts their positions over the course of optimization, leading to improved model learning and inference. We provide theoretical guarantees that our bandit optimization converges to the regret-minimizing choice of integration points. Empirical validation of our algorithm is provided in terms of improved learning and inference in Variational Autoencoders and Sigmoid Belief Networks.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 8

03/12/2018

Variational Inference for Gaussian Process with Panel Count Data

We present the first framework for Gaussian-process-modulated Poisson pr...
03/21/2017

A Deterministic Global Optimization Method for Variational Inference

Variational inference methods for latent variable statistical models hav...
12/05/2019

Scalable Variational Bayesian Kernel Selection for Sparse Gaussian Process Regression

This paper presents a variational Bayesian kernel selection (VBKS) algor...
10/26/2020

Scalable Gaussian Process Variational Autoencoders

Conventional variational autoencoders fail in modeling correlations betw...
11/07/2014

Scalable Variational Gaussian Process Classification

Gaussian process classification is a popular method with a number of app...
11/05/2021

Dual Parameterization of Sparse Variational Gaussian Processes

Sparse variational Gaussian process (SVGP) methods are a common choice f...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.