Structured Black Box Variational Inference for Latent Time Series Models

07/04/2017
by   Robert Bamler, et al.
0

Continuous latent time series models are prevalent in Bayesian modeling; examples include the Kalman filter, dynamic collaborative filtering, or dynamic topic models. These models often benefit from structured, non mean field variational approximations that capture correlations between time steps. Black box variational inference with reparameterization gradients (BBVI) allows us to explore a rich new class of Bayesian non-conjugate latent time series models; however, a naive application of BBVI to such structured variational models would scale quadratically in the number of time steps. We describe a BBVI algorithm analogous to the forward-backward algorithm which instead scales linearly in time. It allows us to efficiently sample from the variational distribution and estimate the gradients of the ELBO. Finally, we show results on the recently proposed dynamic word embedding model, which was trained using our method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/23/2015

Black box variational inference for state space models

Latent variable time-series models are among the most heavily used tools...
research
05/23/2022

Quasi Black-Box Variational Inference with Natural Gradients for Bayesian Learning

We develop an optimization algorithm suitable for Bayesian learning in c...
research
11/07/2015

Hierarchical Variational Models

Black box variational inference allows researchers to easily prototype a...
research
06/22/2019

Scalable Bayesian dynamic covariance modeling with variational Wishart and inverse Wishart processes

We implement gradient-based variational inference routines for Wishart a...
research
10/22/2021

Automatic Guide Generation for Stan via NumPyro

Stan is a very popular probabilistic language with a state-of-the-art HM...
research
09/29/2022

Structured Optimal Variational Inference for Dynamic Latent Space Models

We consider a latent space model for dynamic networks, where our objecti...
research
01/22/2015

A Collaborative Kalman Filter for Time-Evolving Dyadic Processes

We present the collaborative Kalman filter (CKF), a dynamic model for co...

Please sign up or login with your details

Forgot password? Click here to reset