Learning-by-Novel-View-Synthesis for Full-Face Appearance-based 3D Gaze Estimation

by   Jiawei Qin, et al.

Despite recent advances in appearance-based gaze estimation techniques, the need for training data that covers the target head pose and gaze distribution remains a crucial challenge for practical deployment. This work examines a novel approach for synthesizing gaze estimation training data based on monocular 3D face reconstruction. Unlike prior works using multi-view reconstruction, photo-realistic CG models, or generative neural networks, our approach can manipulate and extend the head pose range of existing training data without any additional requirements. We introduce a projective matching procedure to align the reconstructed 3D facial mesh to the camera coordinate system and synthesize face images with accurate gaze labels. We also propose a mask-guided gaze estimation model and data augmentation strategies to further improve the estimation accuracy by taking advantage of the synthetic training data. Experiments using multiple public datasets show that our approach can significantly improve the estimation performance on challenging cross-dataset settings with non-overlapping gaze distributions.



There are no comments yet.


page 1

page 5

page 7


MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation

Learning-based methods are believed to work well for unconstrained gaze ...

Learning to Detect Head Movement in Unconstrained Remote Gaze Estimation in the Wild

Unconstrained remote gaze estimation remains challenging mostly due to i...

Photo-realistic Monocular Gaze Redirection using Generative Adversarial Networks

Gaze redirection is the task of changing the gaze to a desired direction...

Recurrent CNN for 3D Gaze Estimation using Appearance and Shape Cues

Gaze behavior is an important non-verbal cue in social signal processing...

Highly accurate gaze estimation using a consumer RGB-depth sensor

Determining the direction in which a person is looking is an important p...

GazeGAN - Unpaired Adversarial Image Generation for Gaze Estimation

Recent research has demonstrated the ability to estimate gaze on mobile ...

Detection of Abnormal Behavior with Self-Supervised Gaze Estimation

Due to the recent outbreak of COVID-19, many classes, exams, and meeting...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.