Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networks

by   Adria Recasens, et al.

We introduce a saliency-based distortion layer for convolutional neural networks that helps to improve the spatial sampling of input data for a given task. Our differentiable layer can be added as a preprocessing block to existing task networks and trained altogether in an end-to-end fashion. The effect of the layer is to efficiently estimate how to sample from the original data in order to boost task performance. For example, for an image classification task in which the original data might range in size up to several megapixels, but where the desired input images to the task network are much smaller, our layer learns how best to sample from the underlying high resolution data in a manner which preserves task-relevant information better than uniform downsampling. This has the effect of creating distorted, caricature-like intermediate images, in which idiosyncratic elements of the image that improve task performance are zoomed and exaggerated. Unlike alternative approaches such as spatial transformer networks, our proposed layer is inspired by image saliency, computed efficiently from uniformly downsampled data, and degrades gracefully to a uniform sampling strategy under uncertainty. We apply our layer to improve existing networks for the tasks of human gaze estimation and fine-grained object classification. Code for our method is available in: http://github.com/recasens/Saliency-Sampler


page 2

page 4

page 7

page 9

page 11

page 13


Hallucinating Saliency Maps for Fine-Grained Image Classification for Limited Data Domains

Most of the saliency methods are evaluated on their ability to generate ...

Research on insect pest image detection and recognition based on bio-inspired methods

Insect pest recognition is necessary for crop protection in many areas o...

Human peripheral blur is optimal for object recognition

Our eyes sample a disproportionately large amount of information at the ...

SALISA: Saliency-based Input Sampling for Efficient Video Object Detection

High-resolution images are widely adopted for high-performance object de...

Fast End-to-End Trainable Guided Filter

Image processing and pixel-wise dense prediction have been advanced by h...

Unsupervised Foveal Vision Neural Networks with Top-Down Attention

Deep learning architectures are an extremely powerful tool for recognizi...

Differentiable Patch Selection for Image Recognition

Neural Networks require large amounts of memory and compute to process h...

Please sign up or login with your details

Forgot password? Click here to reset