Self-supervised learning of a facial attribute embedding from video

08/21/2018
by   Olivia Wiles, et al.
4

We propose a self-supervised framework for learning facial attributes by simply watching videos of a human face speaking, laughing, and moving over time. To perform this task, we introduce a network, Facial Attributes-Net (FAb-Net), that is trained to embed multiple frames from the same video face-track into a common low-dimensional space. With this approach, we make three contributions: first, we show that the network can leverage information from multiple source frames by predicting confidence/attention masks for each frame; second, we demonstrate that using a curriculum learning regime improves the learned embedding; finally, we demonstrate that the network learns a meaningful face embedding that encodes information about head pose, facial landmarks and facial expression, i.e. facial attributes, without having been supervised with any labelled data. We are comparable or superior to state-of-the-art self-supervised methods on these tasks and approach the performance of supervised methods.

READ FULL TEXT

page 2

page 7

page 9

page 10

research
10/28/2019

Self-supervised learning of class embeddings from video

This work explores how to use self-supervised learning on videos to lear...
research
10/10/2021

Self-Supervised 3D Face Reconstruction via Conditional Estimation

We present a conditional estimation (CEST) framework to learn 3D facial ...
research
11/15/2022

Towards an objective characterization of an individual's facial movements using Self-Supervised Person-Specific-Models

Disentangling facial movements from other facial characteristics, partic...
research
11/24/2022

Pose-disentangled Contrastive Learning for Self-supervised Facial Representation

Self-supervised facial representation has recently attracted increasing ...
research
07/27/2018

X2Face: A network for controlling face generation by using images, audio, and pose codes

The objective of this paper is a neural network model that controls the ...
research
11/12/2022

MARLIN: Masked Autoencoder for facial video Representation LearnINg

This paper proposes a self-supervised approach to learn universal facial...
research
03/03/2019

Self-Supervised Learning of Face Representations for Video Face Clustering

Analyzing the story behind TV series and movies often requires understan...

Please sign up or login with your details

Forgot password? Click here to reset