SynLiDAR: Learning From Synthetic LiDAR Sequential Point Cloud for Semantic Segmentation

by   Aoran Xiao, et al.

Transfer learning from synthetic to real data has been proved an effective way of mitigating data annotation constraints in various computer vision tasks. However, the developments focused on 2D images but lag far behind for 3D point clouds due to the lack of large-scale high-quality synthetic point cloud data and effective transfer methods. We address this issue by collecting SynLiDAR, a synthetic LiDAR point cloud dataset that contains large-scale point-wise annotated point cloud with accurate geometric shapes and comprehensive semantic classes, and designing PCT-Net, a point cloud translation network that aims to narrow down the gap with real-world point cloud data. For SynLiDAR, we leverage graphic tools and professionals who construct multiple realistic virtual environments with rich scene types and layouts where annotated LiDAR points can be generated automatically. On top of that, PCT-Net disentangles synthetic-to-real gaps into an appearance component and a sparsity component and translates SynLiDAR by aligning the two components with real-world data separately. Extensive experiments over multiple data augmentation and semi-supervised semantic segmentation tasks show very positive outcomes - including SynLiDAR can either train better models or reduce real-world annotated data without sacrificing performance, and PCT-Net translated data further improve model performance consistently.


page 1

page 3

page 5


RIU-Net: Embarrassingly simple semantic segmentation of 3D LiDAR point cloud

This paper proposes RIU-Net (for Range-Image U-Net), the adaptation of a...

Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mapping

Paris-CARLA-3D is a dataset of several dense colored point clouds of out...

STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Dataset

Although various 3D datasets with different functions and scales have be...

BuildingNet: Learning to Label 3D Buildings

We introduce BuildingNet: (a) a large-scale dataset of 3D building model...

CNN-based synthesis of realistic high-resolution LiDAR data

This paper presents a novel CNN-based approach for synthesizing high-res...

Real3D-Aug: Point Cloud Augmentation by Placing Real Objects with Occlusion Handling for 3D Detection and Segmentation

Object detection and semantic segmentation with the 3D lidar point cloud...

POEM: 1-bit Point-wise Operations based on Expectation-Maximization for Efficient Point Cloud Processing

Real-time point cloud processing is fundamental for lots of computer vis...

Code Repositories


Synthetic LiDAR sequential point cloud dataset with point-wise annotations

view repo