Scalable Variational Gaussian Process Regression Networks

03/25/2020
by   Shibo Li, et al.
0

Gaussian process regression networks (GPRN) are powerful Bayesian models for multi-output regression, but their inference is intractable. To address this issue, existing methods use a fully factorized structure (or a mixture of such structures) over all the outputs and latent functions for posterior approximation, which, however, can miss the strong posterior dependencies among the latent variables and hurt the inference quality. In addition, the updates of the variational parameters are inefficient and can be prohibitively expensive for a large number of outputs. To overcome these limitations, we propose a scalable variational inference algorithm for GPRN, which not only captures the abundant posterior dependencies but also is much more efficient for massive outputs. We tensorize the output space and introduce tensor/matrix-normal variational posteriors to capture the posterior correlations and to reduce the parameters. We jointly optimize all the parameters and exploit the inherent Kronecker product structure in the variational model evidence lower bound to accelerate the computation. We demonstrate the advantages of our method in several real-world applications.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/03/2017

Structured Variational Inference for Coupled Gaussian Processes

Sparse variational approximations allow for principled and scalable infe...
research
12/11/2018

Encoding prior knowledge in the structure of the likelihood

The inference of deep hierarchical models is problematic due to strong d...
research
08/29/2023

Multi-Response Heteroscedastic Gaussian Process Models and Their Inference

Despite the widespread utilization of Gaussian process models for versat...
research
02/26/2019

Function Space Particle Optimization for Bayesian Neural Networks

While Bayesian neural networks (BNNs) have drawn increasing attention, t...
research
11/22/2019

A Fully Natural Gradient Scheme for Improving Inference of the Heterogeneous Multi-Output Gaussian Process Model

A recent novel extension of multi-output Gaussian processes handles hete...
research
12/15/2022

Output-Dependent Gaussian Process State-Space Model

Gaussian process state-space model (GPSSM) is a fully probabilistic stat...
research
11/20/2015

The Variational Gaussian Process

Variational inference is a powerful tool for approximate inference, and ...

Please sign up or login with your details

Forgot password? Click here to reset