DeepGaze II: Reading fixations from deep features trained on object recognition

by   Matthias Kümmerer, et al.

Here we present DeepGaze II, a model that predicts where people look in images. The model uses the features from the VGG-19 deep neural network trained to identify objects in images. Contrary to other saliency models that use deep features, here we use the VGG features for saliency prediction with no additional fine-tuning (rather, a few readout layers are trained on top of the VGG features to predict saliency). The model is therefore a strong test of transfer learning. After conservative cross-validation, DeepGaze II explains about 87 achieves top performance in area under the curve metrics on the MIT300 hold-out benchmark. These results corroborate the finding from DeepGaze I (which explained 56 on object recognition provide a versatile feature space for performing related visual tasks. We explore the factors that contribute to this success and present several informative image examples. A web service is available to compute model predictions at


page 10

page 11

page 14

page 15


Understanding and Visualizing Deep Visual Saliency Models

Recently, data-driven deep saliency models have achieved high performanc...

Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet

Recent results suggest that state-of-the-art saliency models perform far...

Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling

Since 2014 transfer learning has become the key driver for the improveme...

Saliency for Fine-grained Object Recognition in Domains with Scarce Training Data

This paper investigates the role of saliency to improve the classificati...

OpenSalicon: An Open Source Implementation of the Salicon Saliency Model

In this technical report, we present our publicly downloadable implement...

Visual Saliency Prediction Using a Mixture of Deep Neural Networks

Visual saliency models have recently begun to incorporate deep learning ...

Code Repositories

Please sign up or login with your details

Forgot password? Click here to reset