DeepGaze II: Reading fixations from deep features trained on object recognition

10/05/2016
by   Matthias Kümmerer, et al.
0

Here we present DeepGaze II, a model that predicts where people look in images. The model uses the features from the VGG-19 deep neural network trained to identify objects in images. Contrary to other saliency models that use deep features, here we use the VGG features for saliency prediction with no additional fine-tuning (rather, a few readout layers are trained on top of the VGG features to predict saliency). The model is therefore a strong test of transfer learning. After conservative cross-validation, DeepGaze II explains about 87 achieves top performance in area under the curve metrics on the MIT300 hold-out benchmark. These results corroborate the finding from DeepGaze I (which explained 56 on object recognition provide a versatile feature space for performing related visual tasks. We explore the factors that contribute to this success and present several informative image examples. A web service is available to compute model predictions at http://deepgaze.bethgelab.org.

READ FULL TEXT

page 10

page 11

page 14

page 15

research
03/06/2019

Understanding and Visualizing Deep Visual Saliency Models

Recently, data-driven deep saliency models have achieved high performanc...
research
11/04/2014

Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet

Recent results suggest that state-of-the-art saliency models perform far...
research
05/26/2021

Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling

Since 2014 transfer learning has become the key driver for the improveme...
research
08/01/2018

Saliency for Fine-grained Object Recognition in Domains with Scarce Training Data

This paper investigates the role of saliency to improve the classificati...
research
06/01/2016

OpenSalicon: An Open Source Implementation of the Salicon Saliency Model

In this technical report, we present our publicly downloadable implement...
research
02/01/2017

Visual Saliency Prediction Using a Mixture of Deep Neural Networks

Visual saliency models have recently begun to incorporate deep learning ...
research
07/10/2014

What you need to know about the state-of-the-art computational models of object-vision: A tour through the models

Models of object vision have been of great interest in computer vision a...

Please sign up or login with your details

Forgot password? Click here to reset