Multimodal Across Domains Gaze Target Detection

08/23/2022
by   Francesco Tonini, et al.
7

This paper addresses the gaze target detection problem in single images captured from the third-person perspective. We present a multimodal deep architecture to infer where a person in a scene is looking. This spatial model is trained on the head images of the person-of- interest, scene and depth maps representing rich context information. Our model, unlike several prior art, do not require supervision of the gaze angles, do not rely on head orientation information and/or location of the eyes of person-of-interest. Extensive experiments demonstrate the stronger performance of our method on multiple benchmark datasets. We also investigated several variations of our method by altering joint-learning of multimodal data. Some variations outperform a few prior art as well. First time in this paper, we inspect domain adaption for gaze target detection, and we empower our multimodal network to effectively handle the domain gap across datasets. The code of the proposed method is available at https://github.com/francescotonini/multimodal-across-domains-gaze-target-detection.

READ FULL TEXT

page 5

page 13

page 14

research
07/18/2023

Object-aware Gaze Target Detection

Gaze target detection aims to predict the image location where the perso...
research
07/11/2023

A Modular Multimodal Architecture for Gaze Target Prediction: Application to Privacy-Sensitive Settings

Predicting where a person is looking is a complex task, requiring to und...
research
07/14/2022

Egocentric Scene Understanding via Multimodal Spatial Rectifier

In this paper, we study a problem of egocentric scene understanding, i.e...
research
09/22/2022

MGTR: End-to-End Mutual Gaze Detection with Transformer

People's looking at each other or mutual gaze is ubiquitous in our daily...
research
07/13/2023

Automated Deception Detection from Videos: Using End-to-End Learning Based High-Level Features and Classification Approaches

Deception detection is an interdisciplinary field attracting researchers...
research
10/10/2021

FLAME: Facial Landmark Heatmap Activated Multimodal Gaze Estimation

3D gaze estimation is about predicting the line of sight of a person in ...
research
11/20/2022

Patch-level Gaze Distribution Prediction for Gaze Following

Gaze following aims to predict where a person is looking in a scene, by ...

Please sign up or login with your details

Forgot password? Click here to reset