Recovering Continuous Scene Dynamics from A Single Blurry Image with Events

04/05/2023
by   Zhangyi Cheng, et al.
1

This paper aims at demystifying a single motion-blurred image with events and revealing temporally continuous scene dynamics encrypted behind motion blurs. To achieve this end, an Implicit Video Function (IVF) is learned to represent a single motion blurred image with concurrent events, enabling the latent sharp image restoration of arbitrary timestamps in the range of imaging exposures. Specifically, a dual attention transformer is proposed to efficiently leverage merits from both modalities, i.e., the high temporal resolution of event features and the smoothness of image features, alleviating temporal ambiguities while suppressing the event noise. The proposed network is trained only with the supervision of ground-truth images of limited referenced timestamps. Motion- and texture-guided supervisions are employed simultaneously to enhance restorations of the non-referenced timestamps and improve the overall sharpness. Experiments on synthetic, semi-synthetic, and real-world datasets demonstrate that our proposed method outperforms state-of-the-art methods by a large margin in terms of both objective PSNR and SSIM measurements and subjective evaluations.

READ FULL TEXT

page 2

page 4

page 8

page 9

page 10

page 12

page 14

research
02/27/2023

Learning to Super-Resolve Blurry Images with Events

Super-Resolution from a single motion Blurred image (SRB) is a severely ...
research
09/18/2023

Learning Parallax for Stereo Event-based Motion Deblurring

Due to the extremely low latency, events have been recently exploited to...
research
01/14/2017

Marked Temporal Dynamics Modeling based on Recurrent Neural Network

We are now witnessing the increasing availability of event stream data, ...
research
04/01/2020

Single Image Optical Flow Estimation with an Event Camera

Event cameras are bio-inspired sensors that asynchronously report intens...
research
04/14/2023

Self-Supervised Scene Dynamic Recovery from Rolling Shutter Images and Events

Scene Dynamic Recovery (SDR) by inverting distorted Rolling Shutter (RS)...
research
08/24/2022

Event-based Image Deblurring with Dynamic Motion Awareness

Non-uniform image deblurring is a challenging task due to the lack of te...
research
11/18/2021

3D Lip Event Detection via Interframe Motion Divergence at Multiple Temporal Resolutions

The lip is a dominant dynamic facial unit when a person is speaking. Det...

Please sign up or login with your details

Forgot password? Click here to reset