Image Reconstruction from Events. Why learn it?

12/12/2021
by   Zelin Zhang, et al.
8

Traditional cameras measure image intensity. Event cameras, by contrast, measure per-pixel temporal intensity changes asynchronously. Recovering intensity from events is a popular research topic since the reconstructed images inherit the high dynamic range (HDR) and high-speed properties of events; hence they can be used in many robotic vision applications and to generate slow-motion HDR videos. However, state-of-the-art methods tackle this problem by training an event-to-image recurrent neural network (RNN), which lacks explainability and is difficult to tune. In this work we show, for the first time, how tackling the joint problem of motion and intensity estimation leads us to model event-based image reconstruction as a linear inverse problem that can be solved without training an image reconstruction RNN. Instead, classical and learning-based image priors can be used to solve the problem and remove artifacts from the reconstructed images. The experiments show that the proposed approach generates images with visual quality on par with state-of-the-art methods despite only using data from a short time interval (i.e., without recurrent connections). Our method can also be used to improve the quality of images reconstructed by approaches that first estimate the image Laplacian; here our method can be interpreted as Poisson reconstruction guided by image priors.

READ FULL TEXT

page 4

page 6

page 7

page 11

page 12

page 13

page 15

page 16

research
05/10/2023

HyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworks

Event-based cameras are becoming increasingly popular for their ability ...
research
11/11/2015

A Continuous Max-Flow Approach to Cyclic Field Reconstruction

Reconstruction of an image from noisy data using Markov Random Field the...
research
06/15/2019

High Speed and High Dynamic Range Video with an Event Camera

Event cameras are novel sensors that report brightness changes in the fo...
research
12/03/2019

Learning to Super Resolve Intensity Images from Events

An event camera detects per-pixel intensity difference and produces asyn...
research
06/24/2022

Ev-NeRF: Event Based Neural Radiance Field

We present Ev-NeRF, a Neural Radiance Field derived from event data. Whi...
research
09/06/2023

SADIR: Shape-Aware Diffusion Models for 3D Image Reconstruction

3D image reconstruction from a limited number of 2D images has been a lo...
research
12/18/2018

Hybrid Loss for Learning Single-Image-based HDR Reconstruction

This paper tackles high-dynamic-range (HDR) image reconstruction given o...

Please sign up or login with your details

Forgot password? Click here to reset