DeepAI
Log In Sign Up

Adversarial Generation of Continuous Images

11/24/2020
by   Ivan Skorokhodov, et al.
4

In most existing learning systems, images are typically viewed as 2D pixel arrays. However, in another paradigm gaining popularity, a 2D image is represented as an implicit neural representation (INR) – an MLP that predicts an RGB pixel value given its (x,y) coordinate. In this paper, we propose two novel architectural techniques for building INR-based image decoders: factorized multiplicative modulation and multi-scale INRs, and use them to build a state-of-the-art continuous image GAN. Previous attempts to adapt INRs for image generation were limited to MNIST-like datasets and do not scale to complex real-world data. Our proposed architectural design improves the performance of continuous image generators by x6-40 times and reaches FID scores of 6.27 on LSUN bedroom 256x256 and 16.32 on FFHQ 1024x1024, greatly reducing the gap between continuous image GANs and pixel-based ones. To the best of our knowledge, these are the highest reported scores for an image generator, that consists entirely of fully-connected layers. Apart from that, we explore several exciting properties of INR-based decoders, like out-of-the-box superresolution, meaningful image-space interpolation, accelerated inference of low-resolution images, an ability to extrapolate outside of image boundaries and strong geometric prior. The source code is available at https://github.com/universome/inr-gan

READ FULL TEXT

page 2

page 3

page 4

page 13

page 15

page 16

page 17

page 18

07/17/2020

XingGAN for Person Image Generation

We propose a novel Generative Adversarial Network (XingGAN or CrossingGA...
05/12/2022

D3T-GAN: Data-Dependent Domain Transfer GANs for Few-shot Image Generation

As an important and challenging problem, few-shot image generation aims ...
10/19/2021

CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis

The style-based GAN (StyleGAN) architecture achieved state-of-the-art re...
07/05/2020

Anatomical Data Augmentation via Fluid-based Image Registration

We introduce a fluid-based image augmentation method for medical image a...
04/02/2022

PixelFolder: An Efficient Progressive Pixel Synthesis Network for Image Generation

Pixel synthesis is a promising research paradigm for image generation, w...
11/06/2022

Distilling Representations from GAN Generator via Squeeze and Span

In recent years, generative adversarial networks (GANs) have been an act...
07/23/2022

Contrastive Monotonic Pixel-Level Modulation

Continuous one-to-many mapping is a less investigated yet important task...