Make Your Brief Stroke Real and Stereoscopic: 3D-Aware Simplified Sketch to Portrait Generation

02/14/2023
by   Yasheng Sun, et al.
0

Creating the photo-realistic version of people sketched portraits is useful to various entertainment purposes. Existing studies only generate portraits in the 2D plane with fixed views, making the results less vivid. In this paper, we present Stereoscopic Simplified Sketch-to-Portrait (SSSP), which explores the possibility of creating Stereoscopic 3D-aware portraits from simple contour sketches by involving 3D generative models. Our key insight is to design sketch-aware constraints that can fully exploit the prior knowledge of a tri-plane-based 3D-aware generative model. Specifically, our designed region-aware volume rendering strategy and global consistency constraint further enhance detail correspondences during sketch encoding. Moreover, in order to facilitate the usage of layman users, we propose a Contour-to-Sketch module with vector quantized representations, so that easily drawn contours can directly guide the generation of 3D portraits. Extensive comparisons show that our method generates high-quality results that match the sketch. Our usability study verifies that our system is greatly preferred by user.

READ FULL TEXT

page 2

page 4

page 6

page 7

page 8

page 9

page 12

research
08/20/2019

DeepSketchHair: Deep Sketch-based 3D Hair Modeling

We present sketchhair, a deep learning based tool for interactive modeli...
research
05/30/2023

DiffSketching: Sketch Control Image Synthesis with Diffusion Models

Creative sketch is a universal way of visual expression, but translating...
research
12/06/2021

DoodleFormer: Creative Sketch Drawing with Transformers

Creative sketching or doodling is an expressive activity, where imaginat...
research
09/18/2019

An Unpaired Sketch-to-Photo Translation Model

Sketch-based image synthesis aims to generate a photo image given a sket...
research
06/09/2023

Sketch Beautification: Learning Part Beautification and Structure Refinement for Sketches of Man-made Objects

We present a novel freehand sketch beautification method, which takes as...
research
09/26/2022

A Collaborative, Interactive and Context-Aware Drawing Agent for Co-Creative Design

Recent advances in text-conditioned generative models have provided us w...
research
08/23/2022

DualSmoke: Sketch-Based Smoke Illustration Design with Two-Stage Generative Model

The dynamic effects of smoke are impressive in illustration design, but ...

Please sign up or login with your details

Forgot password? Click here to reset