Speech-Driven Facial Reenactment Using Conditional Generative Adversarial Networks

03/20/2018
by   Seyed Ali Jalalifar, et al.
0

We present a novel approach to generating photo-realistic images of a face with accurate lip sync, given an audio input. By using a recurrent neural network, we achieved mouth landmarks based on audio features. We exploited the power of conditional generative adversarial networks to produce highly-realistic face conditioned on a set of landmarks. These two networks together are capable of producing a sequence of natural faces in sync with an input audio track.

READ FULL TEXT

page 3

page 7

page 9

page 10

research
07/18/2022

Audio Input Generates Continuous Frames to Synthesize Facial Video Using Generative Adiversarial Networks

This paper presents a simple method for speech videos generation based o...
research
09/14/2023

DiffTalker: Co-driven audio-image diffusion for talking faces via intermediate landmarks

Generating realistic talking faces is a complex and widely discussed tas...
research
05/06/2021

Inverting Generative Adversarial Renderer for Face Reconstruction

Given a monocular face image as input, 3D face geometry reconstruction a...
research
02/08/2018

Generating Realistic Geology Conditioned on Physical Measurements with Generative Adversarial Networks

An important problem in geostatistics is to build models of the subsurfa...
research
03/27/2020

Mic2Mic: Using Cycle-Consistent Generative Adversarial Networks to Overcome Microphone Variability in Speech Systems

Mobile and embedded devices are increasingly using microphones and audio...
research
12/11/2017

Eye In-Painting with Exemplar Generative Adversarial Networks

This paper introduces a novel approach to in-painting where the identity...
research
07/17/2020

Personalized Speech2Video with 3D Skeleton Regularization and Expressive Body Poses

In this paper, we propose a novel approach to convert given speech audio...

Please sign up or login with your details

Forgot password? Click here to reset