DeepAI AI Chat
Log In Sign Up

Emergent Properties of Foveated Perceptual Systems

by   Arturo Deza, et al.
Harvard University

We introduce foveated perceptual systems, inspired by human biological systems, and examine the impact that this foveation stage has on the nature and robustness of subsequently learned visual representation. Specifically, these two-stage perceptual systems first foveate an image, inducing a texture-like encoding of peripheral information, which is then inputted to a convolutional neural network (CNN) and trained to perform scene categorization. We find that: 1– Systems trained on foveated inputs (Foveation-Nets) have similar generalization as networks trained on matched-resource networks without foveated input (Standard-Nets), yet show greater cross-generalization. 2– Foveation-Nets show higher robustness than Standard-Nets to scotoma (fovea removed) occlusions, driven by the first foveation stage. 3– Subsequent representations learned in the CNN of Foveation-Nets weigh center information more strongly than Standard-Nets. 4– Foveation-Nets show less sensitivity to low-spatial frequency information than Standard-Nets. Furthermore, when we added biological and artificial augmentation mechanisms to each system through simulated eye-movements or random cropping and mirroring respectively, we found that these effects were amplified. Taken together, we find evidence that foveated perceptual systems learn a visual representation that is distinct from non-foveated perceptual systems, with implications in generalization, robustness, and perceptual sensitivity. These results provide computational support for the idea that the foveated nature of the human visual system might confer a functional advantage for scene representation.


page 2

page 4

page 11


A HVS-inspired Attention Map to Improve CNN-based Perceptual Losses for Image Restoration

Deep Convolutional Neural Network (CNN) features have been demonstrated ...

A Psychovisual Analysis on Deep CNN Features for Perceptual Metrics and A Novel Psychovisual Loss

The efficacy of Deep Convolutional Neural Network (CNN) features as perc...

Eigen-Distortions of Hierarchical Representations

We develop a method for comparing hierarchical image representations in ...

Analysis of Deep Image Quality Models

Subjective image quality measures based on deep neural networks are very...

3D-OOCS: Learning Prostate Segmentation with Inductive Bias

Despite the great success of convolutional neural networks (CNN) in 3D m...

Ultra-fast image categorization in vivo and in silico

Humans are able to robustly categorize images and can, for instance, det...

Code Repositories


An adapted version of the Metamer Foveation Transform code from Deza et al. ICLR 2019

view repo