Three for one and one for three: Flow, Segmentation, and Surface Normals

07/19/2018
by   Hoang-An Le, et al.
0

Optical flow, semantic segmentation, and surface normals represent different information modalities, yet together they bring better cues for scene understanding problems. In this paper, we study the influence between the three modalities: how one impacts on the others and their efficiency in combination. We employ a modular approach using a convolutional refinement network which is trained supervised but isolated from RGB images to enforce joint modality features. To assist the training process, we create a large-scale synthetic outdoor dataset that supports dense annotation of semantic segmentation, optical flow, and surface normals. The experimental results show positive influence among the three modalities, especially for objects' boundaries, region consistency, and scene structures.

READ FULL TEXT

page 2

page 8

page 10

07/26/2016

Joint Optical Flow and Temporally Consistent Semantic Segmentation

The importance and demands of visual scene understanding have been stead...
11/09/2020

EDEN: Multimodal Synthetic Dataset of Enclosed GarDEN Scenes

Multimodal large-scale datasets for outdoor scenes are mostly designed f...
01/11/2019

Optical Flow augmented Semantic Segmentation networks for Automated Driving

Motion is a dominant cue in automated driving systems. Optical flow is t...
11/20/2019

Unsupervised Domain Adaptation by Optical Flow Augmentation in Semantic Segmentation

It is expensive to generate real-life image labels and there is a domain...
03/12/2016

Optical Flow with Semantic Segmentation and Localized Layers

Existing optical flow methods make generic, spatially homogeneous, assum...
08/03/2016

Learning Common and Specific Features for RGB-D Semantic Segmentation with Deconvolutional Networks

In this paper, we tackle the problem of RGB-D semantic segmentation of i...
03/29/2022

Abstract Flow for Temporal Semantic Segmentation on the Permutohedral Lattice

Semantic segmentation is a core ability required by autonomous agents, a...