SegNeRF: 3D Part Segmentation with Neural Radiance Fields

11/21/2022
by   Jesus Zarzar, et al.
0

Recent advances in Neural Radiance Fields (NeRF) boast impressive performances for generative tasks such as novel view synthesis and 3D reconstruction. Methods based on neural radiance fields are able to represent the 3D world implicitly by relying exclusively on posed images. Yet, they have seldom been explored in the realm of discriminative tasks such as 3D part segmentation. In this work, we attempt to bridge that gap by proposing SegNeRF: a neural field representation that integrates a semantic field along with the usual radiance field. SegNeRF inherits from previous works the ability to perform novel view synthesis and 3D reconstruction, and enables 3D part segmentation from a few images. Our extensive experiments on PartNet show that SegNeRF is capable of simultaneously predicting geometry, appearance, and semantic information from posed images, even for unseen objects. The predicted semantic fields allow SegNeRF to achieve an average mIoU of 30.30 for 2D novel view segmentation, and 37.46 boasting competitive performance against point-based methods by using only a few posed images. Additionally, SegNeRF is able to generate an explicit 3D model from a single image of an object taken in the wild, with its corresponding part segmentation.

READ FULL TEXT

page 8

page 9

page 16

page 17

page 18

page 19

research
02/10/2022

PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis

We present PVSeRF, a learning framework that reconstructs neural radianc...
research
09/08/2022

im2nerf: Image to Neural Radiance Field in the Wild

We propose im2nerf, a learning framework that predicts a continuous neur...
research
04/22/2023

Dehazing-NeRF: Neural Radiance Fields from Hazy Images

Neural Radiance Field (NeRF) has received much attention in recent years...
research
11/25/2021

NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenes

We present NeSF, a method for producing 3D semantic fields from posed RG...
research
10/02/2022

Unsupervised Multi-View Object Segmentation Using Radiance Field Propagation

We present radiance field propagation (RFP), a novel approach to segment...
research
04/17/2021

FiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modelling

We investigate the use of Neural Radiance Fields (NeRF) to learn high qu...
research
08/14/2023

S3IM: Stochastic Structural SIMilarity and Its Unreasonable Effectiveness for Neural Fields

Recently, Neural Radiance Field (NeRF) has shown great success in render...

Please sign up or login with your details

Forgot password? Click here to reset