EventSR: From Asynchronous Events to Image Reconstruction, Restoration, and Super-Resolution via End-to-End Adversarial Learning

03/17/2020
by   Lin Wang, et al.
11

Event cameras sense intensity changes and have many advantages over conventional cameras. To take advantage of event cameras, some methods have been proposed to reconstruct intensity images from event streams. However, the outputs are still in low resolution (LR), noisy, and unrealistic. The low-quality outputs stem broader applications of event cameras, where high spatial resolution (HR) is needed as well as high temporal resolution, dynamic range, and no motion blur. We consider the problem of reconstructing and super-resolving intensity images from LR events, when no ground truth (GT) HR images and down-sampling kernels are available. To tackle the challenges, we propose a novel end-to-end pipeline that reconstructs LR images from event streams, enhances the image qualities and upsamples the enhanced images, called EventSR. For the absence of real GT images, our method is primarily unsupervised, deploying adversarial learning. To train EventSR, we create an open dataset including both real-world and simulated scenes. The use of both datasets boosts up the network performance, and the network architectures and various loss functions in each phase help improve the image qualities. The whole pipeline is trained in three phases. While each phase is mainly for one of the three tasks, the networks in earlier phases are fine-tuned by respective loss functions in an end-to-end manner. Experimental results show that EventSR reconstructs high-quality SR images from events for both simulated and real-world data.

READ FULL TEXT

page 1

page 3

page 6

page 7

page 8

research
12/03/2019

Learning to Super Resolve Intensity Images from Events

An event camera detects per-pixel intensity difference and produces asyn...
research
07/16/2020

Event Enhanced High-Quality Image Recovery

With extremely high temporal resolution, event cameras have a large pote...
research
09/18/2023

Learning Parallax for Stereo Event-based Motion Deblurring

Due to the extremely low latency, events have been recently exploited to...
research
02/27/2023

Learning to Super-Resolve Blurry Images with Events

Super-Resolution from a single motion Blurred image (SRB) is a severely ...
research
04/15/2019

Focus Is All You Need: Loss Functions For Event-based Vision

Event cameras are novel vision sensors that output pixel-level brightnes...
research
12/03/2019

EventGAN: Leveraging Large Scale Image Datasets for Event Cameras

Event cameras provide a number of benefits over traditional cameras, suc...
research
11/24/2021

EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation

Event cameras sense per-pixel intensity changes and produce asynchronous...

Please sign up or login with your details

Forgot password? Click here to reset