Improving Sequential Latent Variable Models with Autoregressive Flows

10/07/2020
by   Joseph Marino, et al.
0

We propose an approach for improving sequence modeling based on autoregressive normalizing flows. Each autoregressive transform, acting across time, serves as a moving frame of reference, removing temporal correlations, and simplifying the modeling of higher-level dynamics. This technique provides a simple, general-purpose method for improving sequence modeling, with connections to existing and classical techniques. We demonstrate the proposed approach both with standalone flow-based models and as a component within sequential latent variable models. Results are presented on three benchmark video datasets, where autoregressive flow-based dynamics improve log-likelihood performance over baseline models. Finally, we illustrate the decorrelation and improved generalization properties of using flow-based dynamics.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 7

page 17

page 18

page 19

05/08/2019

Generative Model with Dynamic Linear Flow

Flow-based generative models are a family of exact log-likelihood models...
05/13/2021

PassFlow: Guessing Passwords with Generative Flows

Recent advances in generative machine learning models rekindled research...
05/17/2021

Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics

This paper introduces an alternative approach to sampling from autoregre...
06/17/2020

Towards Recurrent Autoregressive Flow Models

Stochastic processes generated by non-stationary distributions are diffi...
04/09/2021

Flow-based Autoregressive Structured Prediction of Human Motion

A new method is proposed for human motion predition by learning temporal...
06/07/2017

Improving Variational Auto-Encoders using convex combination linear Inverse Autoregressive Flow

In this paper, we propose a new volume-preserving flow and show that it ...
10/19/2020

Hierarchical Autoregressive Modeling for Neural Video Compression

Recent work by Marino et al. (2020) showed improved performance in seque...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.