TimeReplayer: Unlocking the Potential of Event Cameras for Video Interpolation

03/25/2022
by   Weihua He, et al.
5

Recording fast motion in a high FPS (frame-per-second) requires expensive high-speed cameras. As an alternative, interpolating low-FPS videos from commodity cameras has attracted significant attention. If only low-FPS videos are available, motion assumptions (linear or quadratic) are necessary to infer intermediate frames, which fail to model complex motions. Event camera, a new camera with pixels producing events of brightness change at the temporal resolution of μ s (10^-6 second ), is a game-changing device to enable video interpolation at the presence of arbitrarily complex motion. Since event camera is a novel sensor, its potential has not been fulfilled due to the lack of processing algorithms. The pioneering work Time Lens introduced event cameras to video interpolation by designing optical devices to collect a large amount of paired training data of high-speed frames and events, which is too costly to scale. To fully unlock the potential of event cameras, this paper proposes a novel TimeReplayer algorithm to interpolate videos captured by commodity cameras with events. It is trained in an unsupervised cycle-consistent style, canceling the necessity of high-speed training data and bringing the additional ability of video extrapolation. Its state-of-the-art results and demo videos in supplementary reveal the promising future of event-based vision.

READ FULL TEXT

page 1

page 4

page 7

research
03/02/2020

Learning to Deblur and Generate High Frame Rate Video with an Event Camera

Event cameras are bio-inspired cameras which can measure the change of i...
research
01/17/2019

High-speed Video from Asynchronous Camera Array

This paper presents a method for capturing high-speed video using an asy...
research
06/15/2019

High Speed and High Dynamic Range Video with an Event Camera

Event cameras are novel sensors that report brightness changes in the fo...
research
12/03/2014

Event Retrieval Using Motion Barcodes

We introduce a simple and effective method for retrieval of videos showi...
research
03/23/2019

Analysis of Rolling Shutter Effect on ENF based Video Forensics

ENF is a time-varying signal of the frequency of mains electricity in a ...
research
07/31/2019

Video Stitching for Linear Camera Arrays

Despite the long history of image and video stitching research, existing...
research
09/12/2023

A new meteor detection application robust to camera movements

This article presents a new tool for the automatic detection of meteors....

Please sign up or login with your details

Forgot password? Click here to reset