Log In Sign Up

Adversarial Generation of Continuous Images

by   Ivan Skorokhodov, et al.

In most existing learning systems, images are typically viewed as 2D pixel arrays. However, in another paradigm gaining popularity, a 2D image is represented as an implicit neural representation (INR) – an MLP that predicts an RGB pixel value given its (x,y) coordinate. In this paper, we propose two novel architectural techniques for building INR-based image decoders: factorized multiplicative modulation and multi-scale INRs, and use them to build a state-of-the-art continuous image GAN. Previous attempts to adapt INRs for image generation were limited to MNIST-like datasets and do not scale to complex real-world data. Our proposed architectural design improves the performance of continuous image generators by x6-40 times and reaches FID scores of 6.27 on LSUN bedroom 256x256 and 16.32 on FFHQ 1024x1024, greatly reducing the gap between continuous image GANs and pixel-based ones. To the best of our knowledge, these are the highest reported scores for an image generator, that consists entirely of fully-connected layers. Apart from that, we explore several exciting properties of INR-based decoders, like out-of-the-box superresolution, meaningful image-space interpolation, accelerated inference of low-resolution images, an ability to extrapolate outside of image boundaries and strong geometric prior. The source code is available at


page 2

page 3

page 4

page 13

page 15

page 16

page 17

page 18


XingGAN for Person Image Generation

We propose a novel Generative Adversarial Network (XingGAN or CrossingGA...

D3T-GAN: Data-Dependent Domain Transfer GANs for Few-shot Image Generation

As an important and challenging problem, few-shot image generation aims ...

CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis

The style-based GAN (StyleGAN) architecture achieved state-of-the-art re...

Anatomical Data Augmentation via Fluid-based Image Registration

We introduce a fluid-based image augmentation method for medical image a...

PixelFolder: An Efficient Progressive Pixel Synthesis Network for Image Generation

Pixel synthesis is a promising research paradigm for image generation, w...

Distilling Representations from GAN Generator via Squeeze and Span

In recent years, generative adversarial networks (GANs) have been an act...

Contrastive Monotonic Pixel-Level Modulation

Continuous one-to-many mapping is a less investigated yet important task...