Learning Foveated Reconstruction to Preserve Perceived Image Statistics

08/07/2021
by   Luca Surace, et al.
3

Foveated image reconstruction recovers full image from a sparse set of samples distributed according to the human visual system's retinal sensitivity that rapidly drops with eccentricity. Recently, the use of Generative Adversarial Networks was shown to be a promising solution for such a task as they can successfully hallucinate missing image information. Like for other supervised learning approaches, also for this one, the definition of the loss function and training strategy heavily influences the output quality. In this work, we pose the question of how to efficiently guide the training of foveated reconstruction techniques such that they are fully aware of the human visual system's capabilities and limitations, and therefore, reconstruct visually important image features. Due to the nature of GAN-based solutions, we concentrate on the human's sensitivity to hallucination for different input sample densities. We present new psychophysical experiments, a dataset, and a procedure for training foveated image reconstruction. The strategy provides flexibility to the generator network by penalizing only perceptually important deviations in the output. As a result, the method aims to preserve perceived image statistics rather than natural image statistics. We evaluate our strategy and compare it to alternative solutions using a newly trained objective metric and user experiments.

READ FULL TEXT

page 3

page 4

page 7

page 10

page 14

page 15

page 19

page 23

research
01/30/2020

Image Embedded Segmentation: Combining Supervised and Unsupervised Objectives through Generative Adversarial Networks

This paper presents a new regularization method to train a fully convolu...
research
08/06/2018

X-GANs: Image Reconstruction Made Easy for Extreme Cases

Image reconstruction including image restoration and denoising is a chal...
research
05/19/2015

Image Reconstruction from Bag-of-Visual-Words

The objective of this work is to reconstruct an original image from Bag-...
research
06/27/2019

Reconstructing Perceived Images from Brain Activity by Visually-guided Cognitive Representation and Adversarial Learning

Reconstructing perceived images based on brain signals measured with fun...
research
09/12/2019

Perceptual Image Anomaly Detection

We present a novel method for image anomaly detection, where algorithms ...
research
12/18/2019

Frequency-Aware Reconstruction of Fluid Simulations with Generative Networks

Convolutional neural networks were recently employed to fully reconstruc...
research
04/04/2019

Accurate and Fast reconstruction of Porous Media from Extremely Limited Information Using Conditional Generative Adversarial Network

Porous media are ubiquitous in both nature and engineering applications,...

Please sign up or login with your details

Forgot password? Click here to reset