Neural Face Video Compression using Multiple Views

03/29/2022
by   Anna Volokitin, et al.
6

Recent advances in deep generative models led to the development of neural face video compression codecs that use an order of magnitude less bandwidth than engineered codecs. These neural codecs reconstruct the current frame by warping a source frame and using a generative model to compensate for imperfections in the warped source frame. Thereby, the warp is encoded and transmitted using a small number of keypoints rather than a dense flow field, which leads to massive savings compared to traditional codecs. However, by relying on a single source frame only, these methods lead to inaccurate reconstructions (e.g. one side of the head becomes unoccluded when turning the head and has to be synthesized). Here, we aim to tackle this issue by relying on multiple source frames (views of the face) and present encouraging results.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/22/2022

Thinking the Fusion Strategy of Multi-reference Face Reenactment

In recent advances of deep generative models, face reenactment -manipula...
research
09/29/2020

Neural Alignment for Face De-pixelization

We present a simple method to reconstruct a high-resolution video from a...
research
12/01/2020

Low Bandwidth Video-Chat Compression using Deep Generative Models

To unlock video chat for hundreds of millions of people hindered by poor...
research
04/27/2023

Controllable One-Shot Face Video Synthesis With Semantic Aware Prior

The one-shot talking-head synthesis task aims to animate a source image ...
research
03/04/2017

Generative Compression

Traditional image and video compression algorithms rely on hand-crafted ...
research
05/28/2018

Deep Generative Models for Distribution-Preserving Lossy Compression

We propose and study the problem of distribution-preserving lossy compre...
research
04/27/2020

Remote Photoplethysmography: Rarely Considered Factors

Remote Photoplethysmography (rPPG) is a fast-growing technique of vital ...

Please sign up or login with your details

Forgot password? Click here to reset