Stereo Hybrid Event-Frame (SHEF) Cameras for 3D Perception

10/11/2021
by   Ziwei Wang, et al.
0

Stereo camera systems play an important role in robotics applications to perceive the 3D world. However, conventional cameras have drawbacks such as low dynamic range, motion blur and latency due to the underlying frame-based mechanism. Event cameras address these limitations as they report the brightness changes of each pixel independently with a fine temporal resolution, but they are unable to acquire absolute intensity information directly. Although integrated hybrid event-frame sensors (eg., DAVIS) are available, the quality of data is compromised by coupling at the pixel level in the circuit fabrication of such cameras. This paper proposes a stereo hybrid event-frame (SHEF) camera system that offers a sensor modality with separate high-quality pure event and pure frame cameras, overcoming the limitations of each separate sensor and allowing for stereo depth estimation. We provide a SHEF dataset targeted at evaluating disparity estimation algorithms and introduce a stereo disparity estimation algorithm that uses edge information extracted from the event stream correlated with the edge detected in the frame data. Our disparity estimation outperforms the state-of-the-art stereo matching algorithm on the SHEF dataset.

READ FULL TEXT

page 2

page 4

page 5

page 7

page 8

research
07/17/2019

Stereo Event Lifetime and Disparity Estimation for Dynamic Vision Sensors

Event-based cameras are biologically inspired sensors that output asynch...
research
01/30/2018

The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception

Event based cameras are a new passive sensing modality with a number of ...
research
10/18/2020

Movement-induced Priors for Deep Stereo

We propose a method for fusing stereo disparity estimation with movement...
research
03/24/2018

Realtime Time Synchronized Event-based Stereo

In this work, we propose a novel event based stereo method which address...
research
11/01/2022

Self-Supervised Intensity-Event Stereo Matching

Event cameras are novel bio-inspired vision sensors that output pixel-le...
research
03/31/2020

Du^2Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels

Computational stereo has reached a high level of accuracy, but degrades ...
research
11/19/2022

HALSIE - Hybrid Approach to Learning Segmentation by Simultaneously Exploiting Image and Event Modalities

Standard frame-based algorithms fail to retrieve accurate segmentation m...

Please sign up or login with your details

Forgot password? Click here to reset