Log In Sign Up

How close are we to understanding image-based saliency?

by   Matthias Kümmerer, et al.

Within the set of the many complex factors driving gaze placement, the properities of an image that are associated with fixations under free viewing conditions have been studied extensively. There is a general impression that the field is close to understanding this particular association. Here we frame saliency models probabilistically as point processes, allowing the calculation of log-likelihoods and bringing saliency evaluation into the domain of information. We compared the information gain of state-of-the-art models to a gold standard and find that only one third of the explainable spatial information is captured. We additionally provide a principled method to show where and how models fail to capture information in the fixations. Thus, contrary to previous assertions, purely spatial saliency remains a significant challenge.


page 4

page 8

page 9

page 15

page 18


Fixation prediction with a combined model of bottom-up saliency and vanishing point

By predicting where humans look in natural scenes, we can understand how...

Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet

Recent results suggest that state-of-the-art saliency models perform far...

Improving saliency models' predictions of the next fixation with humans' intrinsic cost of gaze shifts

The human prioritization of image regions can be modeled in a time invar...

SalGaze: Personalizing Gaze Estimation Using Visual Saliency

Traditional gaze estimation methods typically require explicit user cali...

How do people explore virtual environments?

Understanding how people explore immersive virtual environments is cruci...

Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling

Since 2014 transfer learning has become the key driver for the improveme...

State-of-the-Art in Human Scanpath Prediction

The last years have seen a surge in models predicting the scanpaths of f...