DDSL: Deep Differentiable Simplex Layer for Learning Geometric Signals

01/30/2019
by   Chiyu "Max" Jiang, et al.
8

We present a Deep Differentiable Simplex Layer (DDSL) for neural networks for geometric deep learning. The DDSL is a differentiable layer compatible with deep neural networks for bridging simplex mesh-based geometry representations (point clouds, line mesh, triangular mesh, tetrahedral mesh) with raster images (e.g., 2D/3D grids). The DDSL uses Non-Uniform Fourier Transform (NUFT) to perform differentiable, efficient, anti-aliased rasterization of simplex-based signals. We present a complete theoretical framework for the process as well as an efficient backpropagation algorithm. Compared to previous differentiable renderers and rasterizers, the DDSL generalizes to arbitrary simplex degrees and dimensions. In particular, we explore its applications to 2D shapes and illustrate two applications of this method: (1) mesh editing and optimization guided by neural network outputs, and (2) using DDSL for a differentiable rasterization loss to facilitate end-to-end training of polygon generators. We are able to validate the effectiveness of gradient-based shape optimization with the example of airfoil optimization, and using the differentiable rasterization loss to facilitate end-to-end training, we surpass state of the art for polygonal image segmentation given ground-truth bounding boxes.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 5

page 8

page 14

06/20/2021

DeepMesh: Differentiable Iso-Surface Extraction

Geometric Deep Learning has recently made striking progress with the adv...
04/29/2021

NURBS-Diff: A Differentiable NURBS Layer for Machine Learning CAD Applications

Recent deep-learning-based techniques for the reconstruction of geometri...
11/21/2021

Differentiable Projection for Constrained Deep Learning

Deep neural networks (DNNs) have achieved extraordinary performance in s...
11/20/2017

Neural 3D Mesh Renderer

For modeling the 3D world behind 2D images, which 3D representation is m...
04/29/2021

MongeNet: Efficient Sampler for Geometric Deep Learning

Recent advances in geometric deep-learning introduce complex computation...
12/01/2019

End to End Trainable Active Contours via Differentiable Rendering

We present an image segmentation method that iteratively evolves a polyg...
07/16/2020

Accelerating 3D Deep Learning with PyTorch3D

Deep learning has significantly improved 2D image recognition. Extending...

Code Repositories

DDSL

DDSL: Deep Differential Simplex Layer for Neural Networks


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.