Context-empowered Visual Attention Prediction in Pedestrian Scenarios

by   Igor Vozniak, et al.

Effective and flexible allocation of visual attention is key for pedestrians who have to navigate to a desired goal under different conditions of urgency and safety preferences. While automatic modelling of pedestrian attention holds great promise to improve simulations of pedestrian behavior, current saliency prediction approaches mostly focus on generic free-viewing scenarios and do not reflect the specific challenges present in pedestrian attention prediction. In this paper, we present Context-SalNET, a novel encoder-decoder architecture that explicitly addresses three key challenges of visual attention prediction in pedestrians: First, Context-SalNET explicitly models the context factors urgency and safety preference in the latent space of the encoder-decoder model. Second, we propose the exponentially weighted mean squared error loss (ew-MSE) that is able to better cope with the fact that only a small part of the ground truth saliency maps consist of non-zero entries. Third, we explicitly model epistemic uncertainty to account for the fact that training data for pedestrian attention prediction is limited. To evaluate Context-SalNET, we recorded the first dataset of pedestrian visual attention in VR that includes explicit variation of the context factors urgency and safety preference. Context-SalNET achieves clear improvements over state-of-the-art saliency prediction approaches as well as over ablations. Our novel dataset will be made fully available and can serve as a valuable resource for further research on pedestrian attention prediction.


page 1

page 4

page 5

page 6


Looking Ahead: Anticipating Pedestrians Crossing with Future Frames Prediction

In this paper, we present an end-to-end future-prediction model that foc...

SalyPath360: Saliency and Scanpath Prediction Framework for Omnidirectional Images

This paper introduces a new framework to predict visual attention of omn...

InCrowdFormer: On-Ground Pedestrian World Model From Egocentric Views

We introduce an on-ground Pedestrian World Model, a computational model ...

Where's YOUR focus: Personalized Attention

Human visual attention is subjective and biased according to the persona...

Context-aware Pedestrian Trajectory Prediction with Multimodal Transformer

We propose a novel solution for predicting future trajectories of pedest...

UACANet: Uncertainty Augmented Context Attention for Polyp Semgnetaion

We propose Uncertainty Augmented Context Attention network (UACANet) for...

Pedestrian Detection with Autoregressive Network Phases

We present an autoregressive pedestrian detection framework with cascade...

Please sign up or login with your details

Forgot password? Click here to reset