DeepSurfels: Learning Online Appearance Fusion

12/28/2020
by   Marko Mihajlovic, et al.
8

We present DeepSurfels, a novel hybrid scene representation for geometry and appearance information. DeepSurfels combines explicit and neural building blocks to jointly encode geometry and appearance information. In contrast to established representations, DeepSurfels better represents high-frequency textures, is well-suited for online updates of appearance information, and can be easily combined with machine learning methods. We further present an end-to-end trainable online appearance fusion pipeline that fuses information provided by RGB images into the proposed scene representation and is trained using self-supervision imposed by the reprojection error with respect to the input images. Our method compares favorably to classical texture mapping approaches as well as recently proposed learning-based techniques. Moreover, we demonstrate lower runtime, improved generalization capabilities, and better scalability to larger scenes compared to existing methods.

READ FULL TEXT

page 2

page 5

page 7

page 8

page 15

page 16

page 17

research
06/04/2019

Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations

The advent of deep learning has given rise to neural scene representatio...
research
11/08/2022

Deep Appearance Prefiltering

Physically based rendering of complex scenes can be prohibitively costly...
research
05/17/2022

Self-supervised Neural Articulated Shape and Appearance Models

Learning geometry, motion, and appearance priors of object classes is im...
research
03/28/2020

Inferring Semantic Information with 3D Neural Scene Representations

Biological vision infers multi-modal 3D representations that support rea...
research
03/24/2023

TEGLO: High Fidelity Canonical Texture Mapping from Single-View Images

Recent work in Neural Fields (NFs) learn 3D representations from class-s...
research
09/19/2023

PanopticNeRF-360: Panoramic 3D-to-2D Label Transfer in Urban Scenes

Training perception systems for self-driving cars requires substantial a...
research
11/24/2017

Appearance-and-Relation Networks for Video Classification

Spatiotemporal feature learning in videos is a fundamental and difficult...

Please sign up or login with your details

Forgot password? Click here to reset