Style and Content Disentanglement in Generative Adversarial Networks

11/14/2018
by   Hadi Kazemi, et al.
0

Disentangling factors of variation within data has become a very challenging problem for image generation tasks. Current frameworks for training a Generative Adversarial Network (GAN), learn to disentangle the representations of the data in an unsupervised fashion and capture the most significant factors of the data variations. However, these approaches ignore the principle of content and style disentanglement in image generation, which means their learned latent code may alter the content and style of the generated images at the same time. This paper describes the Style and Content Disentangled GAN (SC-GAN), a new unsupervised algorithm for training GANs that learns disentangled style and content representations of the data. We assume that the representation of an image can be decomposed into a content code that represents the geometrical information of the data, and a style code that captures textural properties. Consequently, by fixing the style portion of the latent representation, we can generate diverse images in a particular style. Reversely, we can set the content code and generate a specific scene in a variety of styles. The proposed SC-GAN has two components: a content code which is the input to the generator, and a style code which modifies the scene style through modification of the Adaptive Instance Normalization (AdaIN) layers' parameters. We evaluate the proposed SC-GAN framework on a set of baseline datasets.

READ FULL TEXT
research
03/17/2016

Generative Image Modeling using Style and Structure Adversarial Networks

Current generative frameworks use end-to-end learning and generate image...
research
12/08/2020

Conditional Generation of Medical Images via Disentangled Adversarial Inference

Synthetic medical image generation has a huge potential for improving he...
research
07/04/2018

MIXGAN: Learning Concepts from Different Domains for Mixture Generation

In this work, we present an interesting attempt on mixture generation: a...
research
05/11/2019

Disentangling Content and Style via Unsupervised Geometry Distillation

It is challenging to disentangle an object into two orthogonal spaces of...
research
07/23/2021

AD-GAN: End-to-end Unsupervised Nuclei Segmentation with Aligned Disentangling Training

We consider unsupervised cell nuclei segmentation in this paper. Exploit...
research
03/04/2023

Decompose, Adjust, Compose: Effective Normalization by Playing with Frequency for Domain Generalization

Domain generalization (DG) is a principal task to evaluate the robustnes...
research
11/07/2018

Style Separation and Synthesis via Generative Adversarial Networks

Style synthesis attracts great interests recently, while few works focus...

Please sign up or login with your details

Forgot password? Click here to reset