Patch-level Gaze Distribution Prediction for Gaze Following

by   Qiaomu Miao, et al.
Stony Brook University

Gaze following aims to predict where a person is looking in a scene, by predicting the target location, or indicating that the target is located outside the image. Recent works detect the gaze target by training a heatmap regression task with a pixel-wise mean-square error (MSE) loss, while formulating the in/out prediction task as a binary classification task. This training formulation puts a strict, pixel-level constraint in higher resolution on the single annotation available in training, and does not consider annotation variance and the correlation between the two subtasks. To address these issues, we introduce the patch distribution prediction (PDP) method. We replace the in/out prediction branch in previous models with the PDP branch, by predicting a patch-level gaze distribution that also considers the outside cases. Experiments show that our model regularizes the MSE loss by predicting better heatmap distributions on images with larger annotation variances, meanwhile bridging the gap between the target prediction and in/out prediction subtasks, showing a significant improvement in performance on both subtasks on public gaze following datasets.


page 2

page 3

page 7

page 8

page 12

page 13

page 15

page 16


GOO: A Dataset for Gaze Object Prediction in Retail Environments

One of the most fundamental and information-laden actions humans do is t...

Image Aesthetics Prediction Using Multiple Patches Preserving the Original Aspect Ratio of Contents

The spread of social networking services has created an increasing deman...

Object-aware Gaze Target Detection

Gaze target detection aims to predict the image location where the perso...

Following Gaze Across Views

Following the gaze of people inside videos is an important signal for un...

Believe It or Not, We Know What You Are Looking at!

By borrowing the wisdom of human in gaze following, we propose a two-sta...

Multimodal Across Domains Gaze Target Detection

This paper addresses the gaze target detection problem in single images ...

Eyes are the Windows to the Soul: Predicting the Rating of Text Quality Using Gaze Behaviour

Predicting a reader's rating of text quality is a challenging task that ...

Please sign up or login with your details

Forgot password? Click here to reset