Real-Time Semantic Stereo Matching

10/01/2019
by   Pier Luigi Dovesi, et al.
9

Scene understanding is paramount in robotics, self-navigation, augmented reality, and many other fields. To fully accomplish this task, an autonomous agent has to infer the 3D structure of the sensed scene (to know where it looks at) and its content (to know what it sees). To tackle the two tasks, deep neural networks trained to infer semantic segmentation and depth from stereo images are often the preferred choices. Specifically, Semantic Stereo Matching can be tackled by either standalone models trained for the two tasks independently or joint end-to-end architectures. Nonetheless, as proposed so far, both solutions are inefficient because requiring two forward passes in the former case or due to the complexity of a single network in the latter, although jointly tackling both tasks is usually beneficial in terms of accuracy. In this paper, we propose a single compact and lightweight architecture for real-time semantic stereo matching. Our framework relies on coarse-to-fine estimations in a multi-stage fashion, allowing: i) very fast inference even on embedded devices, with marginal drops in accuracy, compared to state-of-the-art networks, ii) trade accuracy for speed, according to the specific application requirements. Experimental results on high-end GPUs as well as on an embedded Jetson TX2 confirm the superiority of semantic stereo matching compared to standalone tasks and highlight the versatility of our framework on any hardware and for any application.

READ FULL TEXT

page 1

page 3

page 6

research
09/21/2021

Survey on Semantic Stereo Matching / Semantic Depth Estimation

Stereo matching is one of the widely used techniques for inferring depth...
research
08/25/2021

Semantic Scene Segmentation for Robotics Applications

Semantic scene segmentation plays a critical role in a wide range of rob...
research
11/22/2019

Learning End-To-End Scene Flow by Distilling Single Tasks Knowledge

Scene flow is a challenging task aimed at jointly estimating the 3D stru...
research
10/12/2018

Real-time self-adaptive deep stereo

Deep convolutional neural networks trained end-to-end are the undisputed...
research
06/05/2020

FP-Stereo: Hardware-Efficient Stereo Vision for Embedded Applications

Fast and accurate depth estimation, or stereo matching, is essential in ...
research
06/07/2021

Multi-Exit Semantic Segmentation Networks

Semantic segmentation arises as the backbone of many vision systems, spa...
research
06/05/2018

Performance Evaluation of Deep Learning Networks for Semantic Segmentation of Traffic Stereo-Pair Images

Semantic image segmentation is one the most demanding task, especially f...

Please sign up or login with your details

Forgot password? Click here to reset