Generative Semantic Manipulation with Contrasting GAN

08/01/2017
by   Xiaodan Liang, et al.
0

Generative Adversarial Networks (GANs) have recently achieved significant improvement on paired/unpaired image-to-image translation, such as photo→ sketch and artist painting style transfer. However, existing models can only be capable of transferring the low-level information (e.g. color or texture changes), but fail to edit high-level semantic meanings (e.g., geometric structure or content) of objects. On the other hand, while some researches can synthesize compelling real-world images given a class label or caption, they cannot condition on arbitrary shapes or structures, which largely limits their application scenarios and interpretive capability of model results. In this work, we focus on a more challenging semantic manipulation task, which aims to modify the semantic meaning of an object while preserving its own characteristics (e.g. viewpoints and shapes), such as cow→sheep, motor→ bicycle, cat→dog. To tackle such large semantic changes, we introduce a contrasting GAN (contrast-GAN) with a novel adversarial contrasting objective. Instead of directly making the synthesized samples close to target data as previous GANs did, our adversarial contrasting objective optimizes over the distance comparisons between samples, that is, enforcing the manipulated data be semantically closer to the real data with target category than the input data. Equipped with the new contrasting objective, a novel mask-conditional contrast-GAN architecture is proposed to enable disentangle image background with object semantic changes. Experiments on several semantic manipulation tasks on ImageNet and MSCOCO dataset show considerable performance gain by our contrast-GAN over other conditional GANs. Quantitative results further demonstrate the superiority of our model on generating manipulated results with high visual fidelity and reasonable object semantics.

READ FULL TEXT

page 2

page 5

page 6

page 7

page 8

page 10

page 11

page 12

research
03/28/2019

Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation

The state-of-the-art approaches in Generative Adversarial Networks (GANs...
research
11/30/2022

Extracting Semantic Knowledge from GANs with Unsupervised Learning

Recently, unsupervised learning has made impressive progress on various ...
research
05/19/2018

Sparsely Grouped Multi-task Generative Adversarial Networks for Facial Attribute Manipulation

Recently, Image-to-Image Translation (IIT) has made great progress in en...
research
05/18/2023

Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold

Synthesizing visual content that meets users' needs often requires flexi...
research
01/05/2018

Semantic-aware Grad-GAN for Virtual-to-Real Urban Scene Adaption

Recent advances in vision tasks (e.g., segmentation) highly depend on th...
research
10/18/2021

Boosting Image Outpainting with Semantic Layout Prediction

The objective of image outpainting is to extend image current border and...
research
05/15/2020

Semantic Photo Manipulation with a Generative Image Prior

Despite the recent success of GANs in synthesizing images conditioned on...

Please sign up or login with your details

Forgot password? Click here to reset