DSM-Net: Disentangled Structured Mesh Net for Controllable Generation of Fine Geometry

08/12/2020
by   Jie Yang, et al.
2

3D shape generation is a fundamental operation in computer graphics. While significant progress has been made, especially with recent deep generative models, it remains a challenge to synthesize high-quality geometric shapes with rich detail and complex structure, in a controllable manner. To tackle this, we introduce DSM-Net, a deep neural network that learns a disentangled structured mesh representation for 3D shapes, where two key aspects of shapes, geometry and structure, are encoded in a synergistic manner to ensure plausibility of the generated shapes, while also being disentangled as much as possible. This supports a range of novel shape generation applications with intuitive control, such as interpolation of structure (geometry) while keeping geometry (structure) unchanged. To achieve this, we simultaneously learn structure and geometry through variational autoencoders (VAEs) in a hierarchical manner for both, with bijective mappings at each level. In this manner we effectively encode geometry and structure in separate latent spaces, while ensuring their compatibility: the structure is used to guide the geometry and vice versa. At the leaf level, the part geometry is represented using a conditional part VAE, to encode high-quality geometric details, guided by the structure context as the condition. Our method not only supports controllable generation applications, but also produces high-quality synthesized shapes, outperforming state-of-the-art methods.

READ FULL TEXT

page 2

page 3

page 4

page 5

page 9

page 18

page 19

page 20

08/01/2019

StructureNet: Hierarchical Graph Networks for 3D Shape Generation

The ability to generate novel, diverse, and realistic 3D shapes along wi...
09/17/2020

ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis

Manually authoring 3D shapes is difficult and time consuming; generative...
08/05/2021

SimpModeling: Sketching Implicit Field to Guide Mesh Modeling for 3D Animalmorphic Head Design

Head shapes play an important role in 3D character design. In this work,...
06/07/2019

Latent feature disentanglement for 3D meshes

Generative modeling of 3D shapes has become an important problem due to ...
03/09/2022

The Transitive Information Theory and its Application to Deep Generative Models

Paradoxically, a Variational Autoencoder (VAE) could be pushed in two op...
05/27/2021

MeshCNN Fundamentals: Geometric Learning through a Reconstructable Representation

Mesh-based learning is one of the popular approaches nowadays to learn s...
06/10/2022

Neural Template: Topology-aware Reconstruction and Disentangled Generation of 3D Meshes

This paper introduces a novel framework called DTNet for 3D mesh reconst...