High-Fidelity Synthesis with Disentangled Representation

01/13/2020
by   Wonkwang Lee, et al.
0

Learning disentangled representation of data without supervision is an important step towards improving the interpretability of generative models. Despite recent advances in disentangled representation learning, existing approaches often suffer from the trade-off between representation learning and generation performance i.e. improving generation quality sacrifices disentanglement performance). We propose an Information-Distillation Generative Adversarial Network (ID-GAN), a simple yet generic framework that easily incorporates the existing state-of-the-art models for both disentanglement learning and high-fidelity synthesis. Our method learns disentangled representation using VAE-based models, and distills the learned representation with an additional nuisance variable to the separate GAN-based generator for high-fidelity synthesis. To ensure that both generative models are aligned to render the same generative factors, we further constrain the GAN generator to maximize the mutual information between the learned latent code and the output. Despite the simplicity, we show that the proposed method is highly effective, achieving comparable image generation quality to the state-of-the-art methods using the disentangled representation. We also show that the proposed decomposition leads to an efficient and stable model design, and we demonstrate photo-realistic high-resolution image synthesis results (1024x1024 pixels) for the first time using the disentangled representations.

READ FULL TEXT

page 8

page 13

page 14

page 15

page 18

page 19

page 20

page 21

04/10/2021

MobileStyleGAN: A Lightweight Convolutional Neural Network for High-Fidelity Image Synthesis

In recent years, the use of Generative Adversarial Networks (GANs) has b...
07/04/2019

Large Scale Adversarial Representation Learning

Adversarially trained generative models (GANs) have recently achieved co...
04/25/2018

Unsupervised Disentangled Representation Learning with Analogical Relations

Learning the disentangled representation of interpretable generative fac...
10/25/2020

Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modelling

Current autoencoder-based disentangled representation learning methods a...
12/20/2021

3D-aware Image Synthesis via Learning Structural and Textural Representations

Making generative models 3D-aware bridges the 2D image space and the 3D ...
06/01/2020

High-Fidelity Audio Generation and Representation Learning with Guided Adversarial Autoencoder

Unsupervised disentangled representation learning from the unlabelled au...
07/04/2021

Controllable cardiac synthesis via disentangled anatomy arithmetic

Acquiring annotated data at scale with rare diseases or conditions remai...