Log In Sign Up

Explicitly Controllable 3D-Aware Portrait Generation

by   Junshu Tang, et al.

In contrast to the traditional avatar creation pipeline which is a costly process, contemporary generative approaches directly learn the data distribution from photographs and the state of the arts can now yield highly photo-realistic images. While plenty of works attempt to extend the unconditional generative models and achieve some level of controllability, it is still challenging to ensure multi-view consistency, especially in large poses. In this work, we propose a 3D portrait generation network that produces 3D consistent portraits while being controllable according to semantic parameters regarding pose, identity, expression and lighting. The generative network uses neural scene representation to model portraits in 3D, whose generation is guided by a parametric face model that supports explicit control. While the latent disentanglement can be further enhanced by contrasting images with partially different attributes, there still exists noticeable inconsistency in non-face areas, e.g., hair and background, when animating expressions. We solve this by proposing a volume blending strategy in which we form a composite output by blending the dynamic and static radiance fields, with two parts segmented from the jointly learned semantic field. Our method outperforms prior arts in extensive experiments, producing realistic portraits with vivid expression in natural lighting when viewed in free viewpoint. The proposed method also demonstrates generalization ability to real images as well as out-of-domain cartoon faces, showing great promise in real applications. Additional video results and code will be available on the project webpage.


page 4

page 9

page 10

page 12

page 13

page 14

page 15

page 16


CONFIG: Controllable Neural Face Image Generation

Our ability to sample realistic natural images, particularly faces, has ...

GIF: Generative Interpretable Faces

Photo-realistic visualization and animation of expressive human faces ha...

Controllable Radiance Fields for Dynamic Face Synthesis

Recent work on 3D-aware image synthesis has achieved compelling results ...

Semantically Controllable Scene Generation with Guidance of Explicit Knowledge

Deep Generative Models (DGMs) are known for their superior capability in...

Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields

Capitalizing on the recent advances in image generation models, existing...

3D GAN Inversion for Controllable Portrait Image Animation

Millions of images of human faces are captured every single day; but the...

StyleRig: Rigging StyleGAN for 3D Control over Portrait Images

StyleGAN generates photorealistic portrait images of faces with eyes, te...