4DContrast: Contrastive Learning with Dynamic Correspondences for 3D Scene Understanding

12/06/2021
by   Yujin Chen, et al.
4

We present a new approach to instill 4D dynamic object priors into learned 3D representations by unsupervised pre-training. We observe that dynamic movement of an object through an environment provides important cues about its objectness, and thus propose to imbue learned 3D representations with such dynamic understanding, that can then be effectively transferred to improved performance in downstream 3D semantic scene understanding tasks. We propose a new data augmentation scheme leveraging synthetic 3D shapes moving in static 3D environments, and employ contrastive learning under 3D-4D constraints that encode 4D invariances into the learned 3D representations. Experiments demonstrate that our unsupervised representation learning results in improvement in downstream 3D semantic segmentation, object detection, and instance segmentation tasks, and moreover, notably improves performance in data-scarce scenarios.

READ FULL TEXT

page 1

page 3

page 7

page 8

page 13

research
04/22/2021

Pri3D: Can 3D Priors Help 2D Representation Learning?

Recent advances in 3D perception have shown impressive progress in under...
research
02/28/2023

Mask3D: Pre-training 2D Vision Transformers by Learning Masked 3D Priors

Current popular backbones in computer vision, such as Vision Transformer...
research
05/30/2022

Self-Supervised Visual Representation Learning with Semantic Grouping

In this paper, we tackle the problem of learning visual representations ...
research
11/16/2022

Keep Your Friends Close Enemies Farther: Debiasing Contrastive Learning with Spatial Priors in 3D Radiology Images

Understanding of spatial attributes is central to effective 3D radiology...
research
06/17/2022

DU-Net based Unsupervised Contrastive Learning for Cancer Segmentation in Histology Images

In this paper, we introduce an unsupervised cancer segmentation framewor...
research
08/04/2020

LoCo: Local Contrastive Representation Learning

Deep neural nets typically perform end-to-end backpropagation to learn t...
research
08/17/2021

RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detection

3D point cloud understanding has made great progress in recent years. Ho...

Please sign up or login with your details

Forgot password? Click here to reset