CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis

10/19/2021
by   Peng Zhou, et al.
13

The style-based GAN (StyleGAN) architecture achieved state-of-the-art results for generating high-quality images, but it lacks explicit and precise control over camera poses. The recently proposed NeRF-based GANs made great progress towards 3D-aware generators, but they are unable to generate high-quality images yet. This paper presents CIPS-3D, a style-based, 3D-aware generator that is composed of a shallow NeRF network and a deep implicit neural representation (INR) network. The generator synthesizes each pixel value independently without any spatial convolution or upsampling operation. In addition, we diagnose the problem of mirror symmetry that implies a suboptimal solution and solve it by introducing an auxiliary discriminator. Trained on raw, single-view images, CIPS-3D sets new records for 3D-aware image synthesis with an impressive FID of 6.97 for images at the 256×256 resolution on FFHQ. We also demonstrate several interesting directions for CIPS-3D such as transfer learning and 3D-aware face stylization. The synthesis results are best viewed as videos, so we recommend the readers to check our github project at https://github.com/PeterouZh/CIPS-3D

READ FULL TEXT

page 1

page 3

page 5

page 7

page 8

research
06/13/2021

Styleformer: Transformer based Generative Adversarial Networks with Style Vector

We propose Styleformer, which is a style-based generator for GAN archite...
research
04/18/2023

Look ATME: The Discriminator Mean Entropy Needs Attention

Generative adversarial networks (GANs) are successfully used for image s...
research
11/27/2020

Image Generators with Conditionally-Independent Pixel Synthesis

Existing image generator networks rely heavily on spatial convolutions a...
research
06/21/2023

Benchmarking and Analyzing 3D-aware Image Synthesis with a Modularized Codebase

Despite the rapid advance of 3D-aware image synthesis, existing studies ...
research
11/24/2020

Adversarial Generation of Continuous Images

In most existing learning systems, images are typically viewed as 2D pix...
research
10/18/2021

StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis

We propose StyleNeRF, a 3D-aware generative model for photo-realistic hi...
research
10/29/2022

Learning Probabilistic Models from Generator Latent Spaces with Hat EBM

This work proposes a method for using any generator network as the found...

Please sign up or login with your details

Forgot password? Click here to reset