Close the Visual Domain Gap by Physics-Grounded Active Stereovision Depth Sensor Simulation

01/28/2022
by   Xiaoshuai Zhang, et al.
0

In this paper, we focus on the simulation of active stereovision depth sensors, which are popular in both academic and industry communities. Inspired by the underlying mechanism of the sensors, we designed a fully physics-grounded simulation pipeline, which includes material acquisition, ray tracing based infrared (IR) image rendering, IR noise simulation, and depth estimation. The pipeline is able to generate depth maps with material-dependent error patterns similar to a real depth sensor. We conduct extensive experiments to show that perception algorithms and reinforcement learning policies trained in our simulation platform could transfer well to real world test cases without any fine-tuning. Furthermore, due to the high degree of realism of this simulation, our depth sensor simulator can be used as a convenient testbed to evaluate the algorithm performance in the real world, which will largely reduce the human effort in developing robotic algorithms. The entire pipeline has been integrated into the SAPIEN simulator and is open-sourced to promote the research of vision and robotics communities.

READ FULL TEXT

page 2

page 5

page 9

page 10

page 11

page 13

page 14

page 16

research
12/24/2020

Simulation of Vision-based Tactile Sensors using Physics based Rendering

Tactile sensing has seen a rapid adoption with the advent of vision-base...
research
12/06/2021

ActiveZero: Mixed Domain Learning for Active Stereovision with Zero Annotation

Traditional depth sensors generate accurate real world depth estimates t...
research
08/04/2020

Reinforced Grounded Action Transformation for Sim-to-Real Transfer

Robots can learn to do complex tasks in simulation, but often, learned b...
research
09/16/2023

OmniLRS: A Photorealistic Simulator for Lunar Robotics

Developing algorithms for extra-terrestrial robotic exploration has alwa...
research
01/05/2021

Monocular Depth Estimation for Soft Visuotactile Sensors

Fluid-filled soft visuotactile sensors such as the Soft-bubbles alleviat...
research
03/08/2023

Aberration-Aware Depth-from-Focus

Computer vision methods for depth estimation usually use simple camera m...
research
02/21/2023

MultiVehicle Simulator (MVSim): lightweight dynamics simulator for multiagents and mobile robotics research

Development of applications related to closed-loop control requires eith...

Please sign up or login with your details

Forgot password? Click here to reset