Ev-NeRF: Event Based Neural Radiance Field

06/24/2022
by   Inwoo Hwang, et al.
0

We present Ev-NeRF, a Neural Radiance Field derived from event data. While event cameras can measure subtle brightness changes in high frame rates, the measurements in low lighting or extreme motion suffer from significant domain discrepancy with complex noise. As a result, the performance of event-based vision tasks does not transfer to challenging environments, where the event cameras are expected to thrive over normal cameras. We find that the multi-view consistency of NeRF provides a powerful self-supervision signal for eliminating the spurious measurements and extracting the consistent underlying structure despite highly noisy input. Instead of posed images of the original NeRF, the input to Ev-NeRF is the event measurements accompanied by the movements of the sensors. Using the loss function that reflects the measurement model of the sensor, Ev-NeRF creates an integrated neural volume that summarizes the unstructured and sparse data points captured for about 2-4 seconds. The generated neural volume can also produce intensity images from novel views with reasonable depth estimates, which can serve as a high-quality input to various vision-based tasks. Our results show that Ev-NeRF achieves competitive performance for intensity image reconstruction under extreme noise conditions and high-dynamic-range imaging.

READ FULL TEXT

page 2

page 4

page 6

page 7

page 8

research
09/28/2020

Learning to Detect Objects with a 1 Megapixel Event Camera

Event cameras encode visual information with high temporal precision, lo...
research
12/15/2022

Event-based Visual Tracking in Dynamic Environments

Visual object tracking under challenging conditions of motion and light ...
research
05/16/2018

Photorealistic Image Reconstruction from Hybrid Intensity and Event based Sensor

Event based sensors encode pixel wise contrast changes as positive or ne...
research
12/12/2021

Image Reconstruction from Events. Why learn it?

Traditional cameras measure image intensity. Event cameras, by contrast,...
research
03/12/2019

Bringing Blurry Alive at High Frame-Rate with an Event Camera

Event-based cameras can measure intensity changes (called ` events') wit...
research
03/03/2021

Event-based Synthetic Aperture Imaging with a Hybrid Network

Synthetic aperture imaging (SAI) is able to achieve the see through effe...
research
03/28/2022

Are High-Resolution Event Cameras Really Needed?

Due to their outstanding properties in challenging conditions, event cam...

Please sign up or login with your details

Forgot password? Click here to reset