PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications

01/19/2017
by   Tim Salimans, et al.
0

PixelCNNs are a recently proposed class of powerful generative models with tractable likelihood. Here we discuss our implementation of PixelCNNs which we make available at https://github.com/openai/pixel-cnn. Our implementation contains a number of modifications to the original model that both simplify its structure and improve its performance. 1) We use a discretized logistic mixture likelihood on the pixels, rather than a 256-way softmax, which we find to speed up training. 2) We condition on whole pixels, rather than R/G/B sub-pixels, simplifying the model structure. 3) We use downsampling to efficiently capture structure at multiple resolutions. 4) We introduce additional short-cut connections to further speed up optimization. 5) We regularize the model using dropout. Finally, we present state-of-the-art log likelihood results on CIFAR-10 to demonstrate the usefulness of these modifications.

READ FULL TEXT

page 5

page 6

page 7

page 9

07/09/2018

Glow: Generative Flow with Invertible 1x1 Convolutions

Flow-based generative models (Dinh et al., 2014) are conceptually attrac...
06/11/2021

Order Matters: Probabilistic Modeling of Node Sequence for Graph Generation

A graph generative model defines a distribution over graphs. One type of...
11/14/2016

On the Quantitative Analysis of Decoder-Based Generative Models

The past several years have seen remarkable progress in generative model...
06/06/2022

FuSS: Fusing Superpixels for Improved Segmentation Consistency

In this work, we propose two different approaches to improve the semanti...
03/25/2022

Efficient-VDVAE: Less is more

Hierarchical VAEs have emerged in recent years as a reliable option for ...
12/28/2017

PixelSNAIL: An Improved Autoregressive Generative Model

Autoregressive generative models consistently achieve the best results i...
08/24/2020

Optimized routines for event generators in QED-PIC codes

In recent years, the prospects of performing fundamental and applied stu...

Code Repositories

pixel-cnn-pp

Pytorch Implementation of PixelCNN++


view repo