WAYLA - Generating Images from Eye Movements

11/21/2017
by   Bingqing Yu, et al.
0

We present a method for reconstructing images viewed by observers based only on their eye movements. By exploring the relationships between gaze patterns and image stimuli, the "What Are You Looking At?" (WAYLA) system learns to synthesize photo-realistic images that are similar to the original pictures being viewed. The WAYLA approach is based on the Conditional Generative Adversarial Network (Conditional GAN) image-to-image translation technique of Isola et al. We consider two specific applications - the first, of reconstructing newspaper images from gaze heat maps, and the second, of detailed reconstruction of images containing only text. The newspaper image reconstruction process is divided into two image-to-image translation operations, the first mapping gaze heat maps into image segmentations, and the second mapping the generated segmentation into a newspaper image. We validate the performance of our approach using various evaluation metrics, along with human visual inspection. All results confirm the ability of our network to perform image generation tasks using eye tracking data.

READ FULL TEXT

page 3

page 6

page 7

research
05/19/2021

Image to Image Translation : Generating maps from satellite images

Generation of maps from satellite images is conventionally done by a ran...
research
01/31/2017

Towards Adversarial Retinal Image Synthesis

Synthesizing images of the eye fundus is a challenging task that has bee...
research
01/21/2019

Generating Text Sequence Images for Recognition

Recently, methods based on deep learning have dominated the field of tex...
research
01/24/2021

cGANs for Cartoon to Real-life Images

The image-to-image translation is a learning task to establish a visual ...
research
02/23/2022

Art Creation with Multi-Conditional StyleGANs

Creating meaningful art is often viewed as a uniquely human endeavor. A ...
research
11/11/2020

Personality-Driven Gaze Animation with Conditional Generative Adversarial Networks

We present a generative adversarial learning approach to synthesize gaze...
research
07/25/2016

DeepWarp: Photorealistic Image Resynthesis for Gaze Manipulation

In this work, we consider the task of generating highly-realistic images...

Please sign up or login with your details

Forgot password? Click here to reset