DeepAI AI Chat
Log In Sign Up

Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images

by   Chao Wen, et al.

We study the problem of shape generation in 3D mesh representation from a small number of color images with or without camera poses. While many previous works learn to hallucinate the shape directly from priors, we adopt to further improve the shape quality by leveraging cross-view information with a graph convolution network. Instead of building a direct mapping function from images to 3D shape, our model learns to predict series of deformations to improve a coarse shape iteratively. Inspired by traditional multiple view geometry methods, our network samples nearby area around the initial mesh's vertex locations and reasons an optimal deformation using perceptual feature statistics built from multiple input images. Extensive experiments show that our model produces accurate 3D shapes that are not only visually plausible from the input perspectives, but also well aligned to arbitrary viewpoints. With the help of physically driven architecture, our model also exhibits generalization capability across different semantic categories, and the number of input images. Model analysis experiments show that our model is robust to the quality of the initial mesh and the error of camera pose, and can be combined with a differentiable renderer for test-time optimization.


page 1

page 4

page 10

page 13

page 14

page 15

page 16


Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation

We study the problem of shape generation in 3D mesh representation from ...

MeshMVS: Multi-View Stereo Guided Mesh Reconstruction

Deep learning based 3D shape generation methods generally utilize latent...

Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images

We propose an end-to-end deep learning architecture that produces a 3D s...

Localization and Mapping using Instance-specific Mesh Models

This paper focuses on building semantic maps, containing object poses an...

Black-Box Test-Time Shape REFINEment for Single View 3D Reconstruction

Much recent progress has been made in reconstructing the 3D shape of an ...

Facetwise Mesh Refinement for Multi-View Stereo

Mesh refinement is a fundamental step for accurate Multi-View Stereo. It...

Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction

We study the problem of recovering an underlying 3D shape from a set of ...