Photo-realistic Monocular Gaze Redirection using Generative Adversarial Networks

by   Zhe He, et al.

Gaze redirection is the task of changing the gaze to a desired direction for a given monocular eye patch image. Many applications such as videoconferencing, films and games, and generation of training data for gaze estimation require redirecting the gaze, without distorting the appearance of the area surrounding the eye and while producing photo-realistic images. Existing methods lack the ability to generate perceptually plausible images. In this work, we present a novel method to alleviate this problem by leveraging generative adversarial training to synthesize an eye image conditioned on a target gaze direction. Our method ensures perceptual similarity and consistency of synthesized images to the real images. Furthermore, a gaze estimation loss is used to control the gaze direction accurately. To attain high-quality images, we incorporate perceptual and cycle consistency losses into our architecture. In extensive evaluations we show that the proposed method outperforms state-of-the-art approaches in terms of both image quality and redirection precision. Finally, we show that generated images can bring significant improvement for the gaze estimation task if used to augment real training data.



There are no comments yet.


page 1

page 6

page 8


Deep Pictorial Gaze Estimation

Estimating human gaze from natural eye images only is a challenging task...

Controllable Continuous Gaze Redirection

In this work, we present interpGaze, a novel framework for controllable ...

Personality-Driven Gaze Animation with Conditional Generative Adversarial Networks

We present a generative adversarial learning approach to synthesize gaze...

ScanGAN360: A Generative Model of Realistic Scanpaths for 360^∘ Images

Understanding and modeling the dynamics of human gaze behavior in 360^∘ ...

MGGR: MultiModal-Guided Gaze Redirection with Coarse-to-Fine Learning

Gaze redirection aims at manipulating a given eye gaze to a desirable di...

Learning-by-Novel-View-Synthesis for Full-Face Appearance-based 3D Gaze Estimation

Despite recent advances in appearance-based gaze estimation techniques, ...

Deep semantic gaze embedding and scanpath comparison for expertise classification during OPT viewing

Modeling eye movement indicative of expertise behavior is decisive in us...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.