DeepAI AI Chat
Log In Sign Up

Image2Point: 3D Point-Cloud Understanding with Pretrained 2D ConvNets

06/08/2021
by   Chenfeng Xu, et al.
Facebook
berkeley college
4

3D point-clouds and 2D images are different visual representations of the physical world. While human vision can understand both representations, computer vision models designed for 2D image and 3D point-cloud understanding are quite different. Our paper investigates the potential for transferability between these two representations by empirically investigating whether this approach works, what factors affect the transfer performance, and how to make it work even better. We discovered that we can indeed use the same neural net model architectures to understand both images and point-clouds. Moreover, we can transfer pretrained weights from image models to point-cloud models with minimal effort. Specifically, based on a 2D ConvNet pretrained on an image dataset, we can transfer the image model to a point-cloud model by inflating 2D convolutional filters to 3D then finetuning its input, output, and optionally normalization layers. The transferred model can achieve competitive performance on 3D point-cloud classification, indoor and driving scene segmentation, even beating a wide range of point-cloud models that adopt task-specific architectures and use a variety of tricks.

READ FULL TEXT

page 17

page 18

page 19

page 20

03/14/2019

Neural Style Transfer for Point Clouds

How can we edit or transform the geometric or color property of a point ...
06/14/2023

Explore In-Context Learning for 3D Point Cloud Understanding

With the rise of large-scale models trained on broad data, in-context le...
09/15/2022

Can We Solve 3D Vision Tasks Starting from A 2D Vision Transformer?

Vision Transformers (ViTs) have proven to be effective, in solving 2D im...
06/11/2023

On the Efficacy of 3D Point Cloud Reinforcement Learning

Recent studies on visual reinforcement learning (visual RL) have explore...
12/05/2022

Images Speak in Images: A Generalist Painter for In-Context Visual Learning

In-context learning, as a new paradigm in NLP, allows the model to rapid...
03/11/2021

An Efficient Hypergraph Approach to Robust Point Cloud Resampling

Efficient processing and feature extraction of largescale point clouds a...
12/03/2021

Bridging the Gap: Point Clouds for Merging Neurons in Connectomics

In the field of Connectomics, a primary problem is that of 3D neuron seg...

Code Repositories

image2point

Official implementation of Image2Point.


view repo