DeepAI AI Chat
Log In Sign Up

Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics

by   Vivek Jayaram, et al.

This paper introduces an alternative approach to sampling from autoregressive models. Autoregressive models are typically sampled sequentially, according to the transition dynamics defined by the model. Instead, we propose a sampling procedure that initializes a sequence with white noise and follows a Markov chain defined by Langevin dynamics on the global log-likelihood of the sequence. This approach parallelizes the sampling process and generalizes to conditional sampling. Using an autoregressive model as a Bayesian prior, we can steer the output of a generative model using a conditional likelihood or constraints. We apply these techniques to autoregressive models in the visual and audio domains, with competitive results for audio source separation, super-resolution, and inpainting.


page 8

page 14

page 15

page 16


Latent Autoregressive Source Separation

Autoregressive models have achieved impressive results over a wide range...

Improving Sequential Latent Variable Models with Autoregressive Flows

We propose an approach for improving sequence modeling based on autoregr...

WaveGrad: Estimating Gradients for Waveform Generation

This paper introduces WaveGrad, a conditional model for waveform generat...

Global Autoregressive Models for Data-Efficient Sequence Learning

Standard autoregressive seq2seq models are easily trained by max-likelih...

ArTIST: Autoregressive Trajectory Inpainting and Scoring for Tracking

One of the core components in online multiple object tracking (MOT) fram...

Learning Discrete Distributions by Dequantization

Media is generally stored digitally and is therefore discrete. Many succ...

Surprisal-Triggered Conditional Computation with Neural Networks

Autoregressive neural network models have been used successfully for seq...