Real-Time Cleaning and Refinement of Facial Animation Signals

by   Eloïse Berson, et al.

With the increasing demand for real-time animated 3D content in the entertainment industry and beyond, performance-based animation has garnered interest among both academic and industrial communities. While recent solutions for motion-capture animation have achieved impressive results, handmade post-processing is often needed, as the generated animations often contain artifacts. Existing real-time motion capture solutions have opted for standard signal processing methods to strengthen temporal coherence of the resulting animations and remove inaccuracies. While these methods produce smooth results, they inherently filter-out part of the dynamics of facial motion, such as high frequency transient movements. In this work, we propose a real-time animation refining system that preserves – or even restores – the natural dynamics of facial motions. To do so, we leverage an off-the-shelf recurrent neural network architecture that learns proper facial dynamics patterns on clean animation data. We parametrize our system using the temporal derivatives of the signal, enabling our network to process animations at any framerate. Qualitative results show that our system is able to retrieve natural motion signals from noisy or degraded input animation.



There are no comments yet.


page 4

page 5


A Robust Interactive Facial Animation Editing System

Over the past few years, the automatic generation of facial animation fo...

High-Quality Real Time Facial Capture Based on Single Camera

We propose a real time deep learning framework for video-based facial ex...

Intuitive Facial Animation Editing Based On A Generative RNN Framework

For the last decades, the concern of producing convincing facial animati...

Real-time Simultaneous 3D Head Modeling and Facial Motion Capture with an RGB-D camera

We propose a method to build in real-time animated 3D head models using ...

A neural network based post-filter for speech-driven head motion synthesis

Despite the fact that neural networks are widely used for speech-driven ...

Motion Artifact Reduction In Photoplethysmography For Reliable Signal Selection

Photoplethysmography (PPG) is a non-invasive and economical technique to...

Real-time 3D Facial Tracking via Cascaded Compositional Learning

We propose to learn a cascade of globally-optimized modular boosted fern...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.