Fine-Grained Off-Road Semantic Segmentation and Mapping via Contrastive Learning

by   Biao Gao, et al.

Road detection or traversability analysis has been a key technique for a mobile robot to traverse complex off-road scenes. The problem has been mainly formulated in early works as a binary classification one, e.g. associating pixels with road or non-road labels. Whereas understanding scenes with fine-grained labels are needed for off-road robots, as scenes are very diverse, and the various mechanical performance of off-road robots may lead to different definitions of safe regions to traverse. How to define and annotate fine-grained labels to achieve meaningful scene understanding for a robot to traverse off-road is still an open question. This research proposes a contrastive learning based method. With a set of human-annotated anchor patches, a feature representation is learned to discriminate regions with different traversability, a method of fine-grained semantic segmentation and mapping is subsequently developed for off-road scene understanding. Experiments are conducted on a dataset of three driving segments that represent very diverse off-road scenes. An anchor accuracy of 89.8 the matching with human-annotated image patches in cross-scene validation. Examined by associated 3D LiDAR data, the fine-grained segments of visual images are demonstrated to have different levels of toughness and terrain elevation, which represents their semantical meaningfulness. The resultant maps contain both fine-grained labels and confidence values, providing rich information to support a robot traversing complex off-road scenes.


page 1

page 3

page 4

page 5

page 6

page 7

page 8


An Active and Contrastive Learning Framework for Fine-Grained Off-Road Semantic Segmentation

Off-road semantic segmentation with fine-grained labels is necessary for...

An Image-based Approach of Task-driven Driving Scene Categorization

Categorizing driving scenes via visual perception is a key technology fo...

TriangleNet: Edge Prior Augmented Network for Semantic Segmentation through Cross-Task Consistency

Semantic segmentation is a classic computer vision problem dedicated to ...

Semantic Clustering for Robust Fine-Grained Scene Recognition

In domain generalization, the knowledge learnt from one or multiple sour...

Learning Fine-Grained Segmentation of 3D Shapes without Part Labels

Learning-based 3D shape segmentation is usually formulated as a semantic...

ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understanding

Recent efforts of multimodal Transformers have improved Visually Rich Do...

PartNet: A Recursive Part Decomposition Network for Fine-grained and Hierarchical Shape Segmentation

Deep learning approaches to 3D shape segmentation are typically formulat...

Please sign up or login with your details

Forgot password? Click here to reset