Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet

11/04/2014
by   Matthias Kümmerer, et al.
0

Recent results suggest that state-of-the-art saliency models perform far from optimal in predicting fixations. This lack in performance has been attributed to an inability to model the influence of high-level image features such as objects. Recent seminal advances in applying deep neural networks to tasks like object recognition suggests that they are able to capture this kind of structure. However, the enormous amount of training data necessary to train these networks makes them difficult to apply directly to saliency prediction. We present a novel way of reusing existing neural networks that have been pretrained on the task of object recognition in models of fixation prediction. Using the well-known network of Krizhevsky et al. (2012), we come up with a new saliency model that significantly outperforms all state-of-the-art models on the MIT Saliency Benchmark. We show that the structure of this network allows new insights in the psychophysics of fixation selection and potentially their neural implementation. To train our network, we build on recent work on the modeling of saliency as point processes.

READ FULL TEXT

page 2

page 8

page 9

research
07/20/2021

Saliency for free: Saliency prediction as a side-effect of object recognition

Saliency is the perceptual capacity of our visual system to focus our at...
research
08/01/2018

Saliency for Fine-grained Object Recognition in Domains with Scarce Training Data

This paper investigates the role of saliency to improve the classificati...
research
10/05/2016

DeepGaze II: Reading fixations from deep features trained on object recognition

Here we present DeepGaze II, a model that predicts where people look in ...
research
05/04/2021

Canonical Saliency Maps: Decoding Deep Face Models

As Deep Neural Network models for face processing tasks approach human-l...
research
01/17/2018

Faster gaze prediction with dense networks and Fisher pruning

Predicting human fixations from images has recently seen large improveme...
research
09/26/2014

How close are we to understanding image-based saliency?

Within the set of the many complex factors driving gaze placement, the p...
research
05/02/2022

Understanding CNNs from excitations

For instance-level explanation, in order to reveal the relations between...

Please sign up or login with your details

Forgot password? Click here to reset