CariGAN: Caricature Generation through Weakly Paired Adversarial Learning

11/01/2018
by   Wenbin Li, et al.
12

Caricature generation is an interesting yet challenging task. The primary goal is to generate plausible caricatures with reasonable exaggerations given face images. Conventional caricature generation approaches mainly use low-level geometric transformations such as image warping to generate exaggerated images, which lack richness and diversity in terms of content and style. The recent progress in generative adversarial networks (GANs) makes it possible to learn an image-to-image transformation from data, so that richer contents and styles can be generated. However, directly applying the GAN-based models to this task leads to unsatisfactory results because there is a large variance in the caricature distribution. Moreover, some models require strictly paired training data which largely limits their usage scenarios. In this paper, we propose CariGAN overcome these problems. Instead of training on paired data, CariGAN learns transformations only from weakly paired images. Specifically, to enforce reasonable exaggeration and facial deformation, facial landmarks are adopted as an additional condition to constrain the generated image. Furthermore, an attention mechanism is introduced to encourage our model to focus on the key facial parts so that more vivid details in these regions can be generated. Finally, a Diversity Loss is proposed to encourage the model to produce diverse results to help alleviate the `mode collapse' problem of the conventional GAN-based models. Extensive experiments on a new large-scale `WebCaricature' dataset show that the proposed CariGAN can generate more plausible caricatures with larger diversity compared with the state-of-the-art models.

READ FULL TEXT

page 1

page 4

page 8

page 9

page 10

research
09/05/2021

VARGAN: Variance Enforcing Network Enhanced GAN

Generative adversarial networks (GANs) are one of the most widely used g...
research
12/22/2021

JoJoGAN: One Shot Face Stylization

While there have been recent advances in few-shot image stylization, the...
research
08/26/2020

Anime-to-Real Clothing: Cosplay Costume Generation via Image-to-Image Translation

Cosplay has grown from its origins at fan conventions into a billion-dol...
research
08/28/2021

Towards Fine-grained Image Classification with Generative Adversarial Networks and Facial Landmark Detection

Fine-grained classification remains a challenging task because distingui...
research
04/17/2017

Multi-View Image Generation from a Single-View

This paper addresses a challenging problem -- how to generate multi-view...
research
02/21/2023

Texturize a GAN Using a Single Image

Can we customize a deep generative model which can generate images that ...
research
12/30/2022

DGFont++: Robust Deformable Generative Networks for Unsupervised Font Generation

Automatic font generation without human experts is a practical and signi...

Please sign up or login with your details

Forgot password? Click here to reset