IMU2Face: Real-time Gesture-driven Facial Reenactment

12/18/2017
by   Justus Thies, et al.
0

We present IMU2Face, a gesture-driven facial reenactment system. To this end, we combine recent advances in facial motion capture and inertial measurement units (IMUs) to control the facial expressions of a person in a target video based on intuitive hand gestures. IMUs are omnipresent, since modern smart-phones, smart-watches and drones integrate such sensors, e.g., for changing the orientation of the screen content, counting steps, or for flight stabilization. Face tracking and reenactment is based on the state-of-the-art real-time Face2Face facial reenactment system. Instead of transferring facial expressions from a source to a target actor, we employ an IMU to track the hand gestures of a source actor and use its orientation to modify the target actor's expressions.

READ FULL TEXT
research
07/29/2020

Face2Face: Real-time Face Capture and Reenactment of RGB Videos

We present Face2Face, a novel approach for real-time facial reenactment ...
research
10/11/2016

FaceVR: Real-Time Facial Reenactment and Eye Gaze Control in Virtual Reality

We introduce FaceVR, a novel method for gaze-aware facial reenactment in...
research
05/10/2018

LSTM-Based Facial Performance Capture Using Embedding Between Expressions

We present a novel end-to-end framework for facial performance capture g...
research
09/05/2019

Neural Style-Preserving Visual Dubbing

Dubbing is a technique for translating video content from one language t...
research
02/08/2016

Automatic Face Reenactment

We propose an image-based, facial reenactment system that replaces the f...
research
06/26/2023

Transfer: Cross Modality Knowledge Transfer using Adversarial Networks – A Study on Gesture Recognition

Knowledge transfer across sensing technology is a novel concept that has...
research
09/30/2021

Audio-Visual Evaluation of Oratory Skills

What makes a talk successful? Is it the content or the presentation? We ...

Please sign up or login with your details

Forgot password? Click here to reset