DeepAI AI Chat
Log In Sign Up

Image2Point: 3D Point-Cloud Understanding with Pretrained 2D ConvNets

by   Chenfeng Xu, et al.
berkeley college

3D point-clouds and 2D images are different visual representations of the physical world. While human vision can understand both representations, computer vision models designed for 2D image and 3D point-cloud understanding are quite different. Our paper investigates the potential for transferability between these two representations by empirically investigating whether this approach works, what factors affect the transfer performance, and how to make it work even better. We discovered that we can indeed use the same neural net model architectures to understand both images and point-clouds. Moreover, we can transfer pretrained weights from image models to point-cloud models with minimal effort. Specifically, based on a 2D ConvNet pretrained on an image dataset, we can transfer the image model to a point-cloud model by inflating 2D convolutional filters to 3D then finetuning its input, output, and optionally normalization layers. The transferred model can achieve competitive performance on 3D point-cloud classification, indoor and driving scene segmentation, even beating a wide range of point-cloud models that adopt task-specific architectures and use a variety of tricks.


page 17

page 18

page 19

page 20


Neural Style Transfer for Point Clouds

How can we edit or transform the geometric or color property of a point ...

Explore In-Context Learning for 3D Point Cloud Understanding

With the rise of large-scale models trained on broad data, in-context le...

Can We Solve 3D Vision Tasks Starting from A 2D Vision Transformer?

Vision Transformers (ViTs) have proven to be effective, in solving 2D im...

On the Efficacy of 3D Point Cloud Reinforcement Learning

Recent studies on visual reinforcement learning (visual RL) have explore...

Images Speak in Images: A Generalist Painter for In-Context Visual Learning

In-context learning, as a new paradigm in NLP, allows the model to rapid...

An Efficient Hypergraph Approach to Robust Point Cloud Resampling

Efficient processing and feature extraction of largescale point clouds a...

Bridging the Gap: Point Clouds for Merging Neurons in Connectomics

In the field of Connectomics, a primary problem is that of 3D neuron seg...

Code Repositories


Official implementation of Image2Point.

view repo