Implicit Posterior Variational Inference for Deep Gaussian Processes

10/26/2019
by   Haibin Yu, et al.
16

A multi-layer deep Gaussian process (DGP) model is a hierarchical composition of GP models with a greater expressive power. Exact DGP inference is intractable, which has motivated the recent development of deterministic and stochastic approximation methods. Unfortunately, the deterministic approximation methods yield a biased posterior belief while the stochastic one is computationally costly. This paper presents an implicit posterior variational inference (IPVI) framework for DGPs that can ideally recover an unbiased posterior belief and still preserve time efficiency. Inspired by generative adversarial networks, our IPVI framework achieves this by casting the DGP inference problem as a two-player game in which a Nash equilibrium, interestingly, coincides with an unbiased posterior belief. This consequently inspires us to devise a best-response dynamics algorithm to search for a Nash equilibrium (i.e., an unbiased posterior belief). Empirical evaluation shows that IPVI outperforms the state-of-the-art approximation methods for DGPs.

READ FULL TEXT
research
04/17/2021

Convolutional Normalizing Flows for Deep Gaussian Processes

Deep Gaussian processes (DGPs), a hierarchical composition of GP models,...
research
03/14/2022

On the Nash equilibrium of moment-matching GANs for stationary Gaussian processes

Generative Adversarial Networks (GANs) learn an implicit generative mode...
research
06/14/2018

Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo

Deep Gaussian Processes (DGPs) are hierarchical generalizations of Gauss...
research
12/05/2019

Scalable Variational Bayesian Kernel Selection for Sparse Gaussian Process Regression

This paper presents a variational Bayesian kernel selection (VBKS) algor...
research
01/31/2014

Neural Variational Inference and Learning in Belief Networks

Highly expressive directed latent variable models, such as sigmoid belie...
research
10/24/2020

Variational Bayesian Unlearning

This paper studies the problem of approximately unlearning a Bayesian mo...
research
06/06/2021

Unbiased Self-Play

We present a general optimization framework for emergent belief-state re...

Please sign up or login with your details

Forgot password? Click here to reset