Monte Carlo Filtering Objectives: A New Family of Variational Objectives to Learn Generative Model and Neural Adaptive Proposal for Time Series

05/20/2021
by   Shuangshuang Chen, et al.
6

Learning generative models and inferring latent trajectories have shown to be challenging for time series due to the intractable marginal likelihoods of flexible generative models. It can be addressed by surrogate objectives for optimization. We propose Monte Carlo filtering objectives (MCFOs), a family of variational objectives for jointly learning parametric generative models and amortized adaptive importance proposals of time series. MCFOs extend the choices of likelihood estimators beyond Sequential Monte Carlo in state-of-the-art objectives, possess important properties revealing the factors for the tightness of objectives, and allow for less biased and variant gradient estimates. We demonstrate that the proposed MCFOs and gradient estimations lead to efficient and stable model learning, and learned generative models well explain data and importance proposals are more sample efficient on various kinds of time series data.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 5

page 6

page 18

page 22

page 23

01/26/2022

Uphill Roads to Variational Tightness: Monotonicity and Monte Carlo Objectives

We revisit the theory of importance weighted variational inference (IWVI...
09/20/2019

Particle Smoothing Variational Objectives

A body of recent work has focused on constructing a variational family o...
06/04/2019

Streaming Variational Monte Carlo

Nonlinear state-space models are powerful tools to describe dynamical st...
03/29/2021

Variational Rejection Particle Filtering

We present a variational inference (VI) framework that unifies and lever...
06/22/2021

Discrepancy-based Inference for Intractable Generative Models using Quasi-Monte Carlo

Intractable generative models are models for which the likelihood is una...
09/23/2015

Deep Temporal Sigmoid Belief Networks for Sequence Modeling

Deep dynamic generative models are developed to learn sequential depende...
12/20/2013

Learning Generative Models with Visual Attention

Attention has long been proposed by psychologists as important for effec...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.