Predictive Sampling with Forecasting Autoregressive Models

02/23/2020
by   Auke J. Wiggers, et al.
52

Autoregressive models (ARMs) currently hold state-of-the-art performance in likelihood-based modeling of image and audio data. Generally, neural network based ARMs are designed to allow fast inference, but sampling from these models is impractically slow. In this paper, we introduce the predictive sampling algorithm: a procedure that exploits the fast inference property of ARMs in order to speed up sampling, while keeping the model intact. We propose two variations of predictive sampling, namely sampling with ARM fixed-point iteration and learned forecasting modules. Their effectiveness is demonstrated in two settings: i) explicit likelihood modeling on binary MNIST, SVHN and CIFAR10, and ii) discrete latent modeling in an autoencoder trained on SVHN, CIFAR10 and Imagenet32. Empirically, we show considerable improvements over baselines in number of ARM inference calls and sampling speed.

READ FULL TEXT

page 6

page 8

page 12

page 13

research
05/26/2022

Training and Inference on Any-Order Autoregressive Models the Right Way

Conditional inference on arbitrary subsets of variables is a core proble...
research
05/28/2018

Discrete flow posteriors for variational inference in discrete dynamical systems

Each training step for a variational autoencoder (VAE) requires us to sa...
research
06/13/2022

Top Two Algorithms Revisited

Top Two algorithms arose as an adaptation of Thompson sampling to best a...
research
02/26/2023

Embodied Self-Supervised Learning (EMSSL) with Sampling and Training Coordination for Robot Arm Inverse Kinematics Model Learning

Forward and inverse kinematics models are fundamental to robot arms, ser...
research
01/17/2021

TSEC: a framework for online experimentation under experimental constraints

Thompson sampling is a popular algorithm for solving multi-armed bandit ...
research
04/20/2017

Fast Generation for Convolutional Autoregressive Models

Convolutional autoregressive models have recently demonstrated state-of-...
research
10/12/2022

Predictive Querying for Autoregressive Neural Sequence Models

In reasoning about sequential events it is natural to pose probabilistic...

Please sign up or login with your details

Forgot password? Click here to reset