ActiveZero: Mixed Domain Learning for Active Stereovision with Zero Annotation

12/06/2021
by   Isabella Liu, et al.
0

Traditional depth sensors generate accurate real world depth estimates that surpass even the most advanced learning approaches trained only on simulation domains. Since ground truth depth is readily available in the simulation domain but quite difficult to obtain in the real domain, we propose a method that leverages the best of both worlds. In this paper we present a new framework, ActiveZero, which is a mixed domain learning solution for active stereovision systems that requires no real world depth annotation. First, we demonstrate the transferability of our method to out-of-distribution real data by using a mixed domain learning strategy. In the simulation domain, we use a combination of supervised disparity loss and self-supervised losses on a shape primitives dataset. By contrast, in the real domain, we only use self-supervised losses on a dataset that is out-of-distribution from either training simulation data or test real data. Second, our method introduces a novel self-supervised loss called temporal IR reprojection to increase the robustness and accuracy of our reprojections in hard-to-perceive regions. Finally, we show how the method can be trained end-to-end and that each module is important for attaining the end result. Extensive qualitative and quantitative evaluations on real data demonstrate state of the art results that can even beat a commercial depth sensor.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

page 12

page 13

research
07/16/2018

ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems

In this paper we present ActiveStereoNet, the first deep learning soluti...
research
03/19/2023

Fully Self-Supervised Depth Estimation from Defocus Clue

Depth-from-defocus (DFD), modeling the relationship between depth and de...
research
12/01/2020

Sim2Real for Self-Supervised Monocular Depth and Segmentation

Image-based learning methods for autonomous vehicle perception tasks req...
research
01/28/2022

Close the Visual Domain Gap by Physics-Grounded Active Stereovision Depth Sensor Simulation

In this paper, we focus on the simulation of active stereovision depth s...
research
10/07/2021

Self-Supervised Depth Completion for Active Stereo

Active stereo systems are widely used in the robotics industry due to th...
research
05/16/2020

Deep feature fusion for self-supervised monocular depth prediction

Recent advances in end-to-end unsupervised learning has significantly im...
research
10/18/2021

SentimentArcs: A Novel Method for Self-Supervised Sentiment Analysis of Time Series Shows SOTA Transformers Can Struggle Finding Narrative Arcs

SOTA Transformer and DNN short text sentiment classifiers report over 97...

Please sign up or login with your details

Forgot password? Click here to reset