Unsupervised Contrastive Photo-to-Caricature Translation based on Auto-distortion

by   Yuhe Ding, et al.

Photo-to-caricature translation aims to synthesize the caricature as a rendered image exaggerating the features through sketching, pencil strokes, or other artistic drawings. Style rendering and geometry deformation are the most important aspects in photo-to-caricature translation task. To take both into consideration, we propose an unsupervised contrastive photo-to-caricature translation architecture. Considering the intuitive artifacts in the existing methods, we propose a contrastive style loss for style rendering to enforce the similarity between the style of rendered photo and the caricature, and simultaneously enhance its discrepancy to the photos. To obtain an exaggerating deformation in an unpaired/unsupervised fashion, we propose a Distortion Prediction Module (DPM) to predict a set of displacements vectors for each input image while fixing some controlling points, followed by the thin plate spline interpolation for warping. The model is trained on unpaired photo and caricature while can offer bidirectional synthesizing via inputting either a photo or a caricature. Extensive experiments demonstrate that the proposed model is effective to generate hand-drawn like caricatures compared with existing competitors.


page 3

page 6

page 7


CariGANs: Unpaired Photo-to-Caricature Translation

Facial caricature is an art form of drawing faces in an exaggerated way ...

An Unpaired Sketch-to-Photo Translation Model

Sketch-based image synthesis aims to generate a photo image given a sket...

Unsupervised Scene Sketch to Photo Synthesis

Sketches make an intuitive and powerful visual expression as they are fa...

Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation

Expanding an existing tourist photo from a partially captured scene to a...

One-Shot Mutual Affine-Transfer for Photorealistic Stylization

Photorealistic style transfer aims to transfer the style of a reference ...

CartoonRenderer: An Instance-based Multi-Style Cartoon Image Translator

Instance based photo cartoonization is one of the challenging image styl...

Unsupervised Multimodal Video-to-Video Translation via Self-Supervised Learning

Existing unsupervised video-to-video translation methods fail to produce...