CtlGAN: Few-shot Artistic Portraits Generation with Contrastive Transfer Learning

03/16/2022
by   Yue Wang, et al.
0

Generating artistic portraits is a challenging problem in computer vision. Existing portrait stylization models that generate good quality results are based on Image-to-Image Translation and require abundant data from both source and target domains. However, without enough data, these methods would result in overfitting. In this work, we propose CtlGAN, a new few-shot artistic portraits generation model with a novel contrastive transfer learning strategy. We adapt a pretrained StyleGAN in the source domain to a target artistic domain with no more than 10 artistic faces. To reduce overfitting to the few training examples, we introduce a novel Cross-Domain Triplet loss which explicitly encourages the target instances generated from different latent codes to be distinguishable. We propose a new encoder which embeds real faces into Z+ space and proposes a dual-path training strategy to better cope with the adapted decoder and eliminate the artifacts. Extensive qualitative, quantitative comparisons and a user study show our method significantly outperforms state-of-the-arts under 10-shot and 1-shot settings and generates high quality artistic portraits. The code will be made publicly available.

READ FULL TEXT

page 14

page 15

page 16

page 22

page 24

page 25

page 27

page 36

research
04/13/2021

Few-shot Image Generation via Cross-domain Correspondence

Training generative models, such as GANs, on a target domain containing ...
research
02/22/2023

ACE: Zero-Shot Image to Image Translation via Pretrained Auto-Contrastive-Encoder

Image-to-image translation is a fundamental task in computer vision. It ...
research
05/13/2021

TransferI2I: Transfer Learning for Image-to-Image Translation from Small Datasets

Image-to-image (I2I) translation has matured in recent years and is able...
research
12/04/2020

Few-shot Image Generation with Elastic Weight Consolidation

Few-shot image generation seeks to generate more data of a given domain,...
research
05/11/2023

WeditGAN: Few-shot Image Generation via Latent Space Relocation

In few-shot image generation, directly training GAN models on just a han...
research
10/22/2021

Pixel-by-Pixel Cross-Domain Alignment for Few-Shot Semantic Segmentation

In this paper we consider the task of semantic segmentation in autonomou...
research
05/08/2022

A Closer Look at Few-shot Image Generation

Modern GANs excel at generating high quality and diverse images. However...

Please sign up or login with your details

Forgot password? Click here to reset