Semi-synthesis: A fast way to produce effective datasets for stereo matching

01/26/2021
by   Ju He, et al.
13

Stereo matching is an important problem in computer vision which has drawn tremendous research attention for decades. Recent years, data-driven methods with convolutional neural networks (CNNs) are continuously pushing stereo matching to new heights. However, data-driven methods require large amount of training data, which is not an easy task for real stereo data due to the annotation difficulties of per-pixel ground-truth disparity. Though synthetic dataset is proposed to fill the gaps of large data demand, the fine-tuning on real dataset is still needed due to the domain variances between synthetic data and real data. In this paper, we found that in synthetic datasets, close-to-real-scene texture rendering is a key factor to boost up stereo matching performance, while close-to-real-scene 3D modeling is less important. We then propose semi-synthetic, an effective and fast way to synthesize large amount of data with close-to-real-scene texture to minimize the gap between synthetic data and real data. Extensive experiments demonstrate that models trained with our proposed semi-synthetic datasets achieve significantly better performance than with general synthetic datasets, especially on real data benchmarks with limited training data. With further fine-tuning on the real dataset, we also achieve SOTA performance on Middlebury and competitive results on KITTI and ETH3D datasets.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 7

research
06/15/2021

Achieving Domain Robustness in Stereo Matching Networks by Removing Shortcut Learning

Learning-based stereo matching and depth estimation networks currently e...
research
11/30/2021

PlantStereo: A Stereo Matching Benchmark for Plant Surface Dense Reconstruction

Stereo matching is an important task in computer vision which has drawn ...
research
03/08/2021

Look, Evolve and Mold: Learning 3D Shape Manifold via Single-view Synthetic Data

With daily observation and prior knowledge, it is easy for us human to i...
research
05/22/2018

Deep Learning with Cinematic Rendering - Fine-Tuning Deep Neural Networks Using Photorealistic Medical Images

Deep learning has emerged as a powerful artificial intelligence tool to ...
research
11/17/2020

A Method to Generate High Precision Mesh Model and RGB-D Datasetfor 6D Pose Estimation Task

Recently, 3D version has been improved greatly due to the development of...
research
01/06/2022

ITSA: An Information-Theoretic Approach to Automatic Shortcut Avoidance and Domain Generalization in Stereo Matching Networks

State-of-the-art stereo matching networks trained only on synthetic data...
research
05/27/2022

Fine-tuning deep learning models for stereo matching using results from semi-global matching

Deep learning (DL) methods are widely investigated for stereo image matc...

Please sign up or login with your details

Forgot password? Click here to reset