A Learned Stereo Depth System for Robotic Manipulation in Homes

09/23/2021
by   Krishna Shankar, et al.
0

We present a passive stereo depth system that produces dense and accurate point clouds optimized for human environments, including dark, textureless, thin, reflective and specular surfaces and objects, at 2560x2048 resolution, with 384 disparities, in 30 ms. The system consists of an algorithm combining learned stereo matching with engineered filtering, a training and data-mixing methodology, and a sensor hardware design. Our architecture is 15x faster than approaches that perform similarly on the Middlebury and Flying Things Stereo Benchmarks. To effectively supervise the training of this model, we combine real data labelled using off-the-shelf depth sensors, as well as a number of different rendered, simulated labeled datasets. We demonstrate the efficacy of our system by presenting a large number of qualitative results in the form of depth maps and point-clouds, experiments validating the metric accuracy of our system and comparisons to other sensors on challenging objects and scenes. We also show the competitiveness of our algorithm compared to state-of-the-art learned models using the Middlebury and FlyingThings datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 6

page 7

research
05/17/2022

Efficient Stereo Depth Estimation for Pseudo LiDAR: A Self-Supervised Approach Based on Multi-Input ResNet Encoder

Perception and localization are essential for autonomous delivery vehicl...
research
07/30/2021

High-Resolution Depth Maps Based on TOF-Stereo Fusion

The combination of range sensors with color cameras can be very useful f...
research
10/07/2021

Self-Supervised Depth Completion for Active Stereo

Active stereo systems are widely used in the robotics industry due to th...
research
08/23/2023

Select-and-Combine (SAC): A Novel Multi-Stereo Depth Fusion Algorithm for Point Cloud Generation via Efficient Local Markov Netlets

Many practical systems for image-based surface reconstruction employ a s...
research
03/29/2021

Detecting and Mapping Trees in Unstructured Environments with a Stereo Camera and Pseudo-Lidar

We present a method for detecting and mapping trees in noisy stereo came...
research
04/08/2019

Noise-Aware Unsupervised Deep Lidar-Stereo Fusion

In this paper, we present LidarStereoNet, the first unsupervised Lidar-s...
research
12/20/2022

360^∘ Stereo Image Composition with Depth Adaption

360^∘ images and videos have become an economic and popular way to provi...

Please sign up or login with your details

Forgot password? Click here to reset