Unsupervised Contrastive Photo-to-Caricature Translation based on Auto-distortion

11/10/2020
by   Yuhe Ding, et al.
0

Photo-to-caricature translation aims to synthesize the caricature as a rendered image exaggerating the features through sketching, pencil strokes, or other artistic drawings. Style rendering and geometry deformation are the most important aspects in photo-to-caricature translation task. To take both into consideration, we propose an unsupervised contrastive photo-to-caricature translation architecture. Considering the intuitive artifacts in the existing methods, we propose a contrastive style loss for style rendering to enforce the similarity between the style of rendered photo and the caricature, and simultaneously enhance its discrepancy to the photos. To obtain an exaggerating deformation in an unpaired/unsupervised fashion, we propose a Distortion Prediction Module (DPM) to predict a set of displacements vectors for each input image while fixing some controlling points, followed by the thin plate spline interpolation for warping. The model is trained on unpaired photo and caricature while can offer bidirectional synthesizing via inputting either a photo or a caricature. Extensive experiments demonstrate that the proposed model is effective to generate hand-drawn like caricatures compared with existing competitors.

READ FULL TEXT

page 3

page 6

page 7

11/01/2018

CariGANs: Unpaired Photo-to-Caricature Translation

Facial caricature is an art form of drawing faces in an exaggerated way ...
09/18/2019

An Unpaired Sketch-to-Photo Translation Model

Sketch-based image synthesis aims to generate a photo image given a sket...
09/06/2022

Unsupervised Scene Sketch to Photo Synthesis

Sketches make an intuitive and powerful visual expression as they are fa...
07/14/2022

Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation

Expanding an existing tourist photo from a partially captured scene to a...
07/24/2019

One-Shot Mutual Affine-Transfer for Photorealistic Stylization

Photorealistic style transfer aims to transfer the style of a reference ...
11/14/2019

CartoonRenderer: An Instance-based Multi-Style Cartoon Image Translator

Instance based photo cartoonization is one of the challenging image styl...
04/14/2020

Unsupervised Multimodal Video-to-Video Translation via Self-Supervised Learning

Existing unsupervised video-to-video translation methods fail to produce...