Intrinsic Neural Fields: Learning Functions on Manifolds

03/15/2022
by   Lukas Koestler, et al.
0

Neural fields have gained significant attention in the computer vision community due to their excellent performance in novel view synthesis, geometry reconstruction, and generative modeling. Some of their advantages are a sound theoretic foundation and an easy implementation in current deep learning frameworks. While neural fields have been applied to signals on manifolds, e.g., for texture reconstruction, their representation has been limited to extrinsically embedding the shape into Euclidean space. The extrinsic embedding ignores known intrinsic manifold properties and is inflexible wrt. transfer of the learned function. To overcome these limitations, this work introduces intrinsic neural fields, a novel and versatile representation for neural fields on manifolds. Intrinsic neural fields combine the advantages of neural fields with the spectral properties of the Laplace-Beltrami operator. We show theoretically that intrinsic neural fields inherit many desirable properties of the extrinsic neural field framework but exhibit additional intrinsic qualities, like isometry invariance. In experiments, we show intrinsic neural fields can reconstruct high-fidelity textures from images with state-of-the-art quality and are robust to the discretization of the underlying manifold. We demonstrate the versatility of intrinsic neural fields by tackling various applications: texture transfer between deformed shapes different shapes, texture reconstruction from real-world images with view dependence, and discretization-agnostic learning on meshes and point clouds.

READ FULL TEXT

page 7

page 11

page 14

page 30

research
02/16/2023

Intrinsic and extrinsic deep learning on manifolds

We propose extrinsic and intrinsic deep neural network architectures as ...
research
05/17/2019

Texture Fields: Learning Texture Representations in Function Space

In recent years, substantial progress has been achieved in learning-base...
research
08/18/2023

Leveraging Intrinsic Properties for Non-Rigid Garment Alignment

We address the problem of aligning real-world 3D data of garments, which...
research
01/28/2022

From data to functa: Your data point is a function and you should treat it like one

It is common practice in deep learning to represent a measurement of the...
research
02/28/2023

IntrinsicNGP: Intrinsic Coordinate based Hash Encoding for Human NeRF

Recently, many works have been proposed to utilize the neural radiance f...
research
01/31/2022

NeuralTailor: Reconstructing Sewing Pattern Structures from 3D Point Clouds of Garments

The fields of SocialVR, performance capture, and virtual try-on are ofte...
research
06/01/2018

Intrinsic Isometric Manifold Learning with Application to Localization

Data living on manifolds commonly appear in many applications. We show t...

Please sign up or login with your details

Forgot password? Click here to reset