GANimation: Anatomically-aware Facial Animation from a Single Image

by   Albert Pumarola, et al.

Recent advances in Generative Adversarial Networks (GANs) have shown impressive results for task of facial expression synthesis. The most successful architecture is StarGAN, that conditions GANs generation process with images of a specific domain, namely a set of images of persons sharing the same expression. While effective, this approach can only generate a discrete number of expressions, determined by the content of the dataset. To address this limitation, in this paper, we introduce a novel GAN conditioning scheme based on Action Units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression. Our approach allows controlling the magnitude of activation of each AU and combine several of them. Additionally, we propose a fully unsupervised strategy to train the model, that only requires images annotated with their activated AUs, and exploit attention mechanisms that make our network robust to changing backgrounds and lighting conditions. Extensive evaluation show that our approach goes beyond competing conditional generators both in the capability to synthesize a much wider range of expressions ruled by anatomically feasible muscle movements, as in the capacity of dealing with images in the wild.


page 2

page 6

page 9

page 10

page 11

page 12

page 13

page 14


Pixel-based Facial Expression Synthesis

Facial expression synthesis has achieved remarkable advances with the ad...

Region Based Adversarial Synthesis of Facial Action Units

Facial expression synthesis or editing has recently received increasing ...

Expression Conditional GAN for Facial Expression-to-Expression Translation

In this paper, we focus on the facial expression translation task and pr...

Uncover Common Facial Expressions in Terracotta Warriors: A Deep Learning Approach

Can advanced deep learning technologies be applied to analyze some ancie...

Novel Deep Learning Approach to Derive Cytokeratin Expression and Epithelium Segmentation from DAPI

Generative Adversarial Networks (GANs) are state of the art for image sy...

A Unified Architecture of Semantic Segmentation and Hierarchical Generative Adversarial Networks for Expression Manipulation

Editing facial expressions by only changing what we want is a long-stand...

Interpretable Deep Neural Networks for Facial Expression and Dimensional Emotion Recognition in-the-wild

In this project, we created a database with two types of annotations use...

Please sign up or login with your details

Forgot password? Click here to reset