Adversarial Texture Optimization from RGB-D Scans

by   Jingwei Huang, et al.

Realistic color texture generation is an important step in RGB-D surface reconstruction, but remains challenging in practice due to inaccuracies in reconstructed geometry, misaligned camera poses, and view-dependent imaging artifacts. In this work, we present a novel approach for color texture generation using a conditional adversarial loss obtained from weakly-supervised views. Specifically, we propose an approach to produce photorealistic textures for approximate surfaces, even from misaligned images, by learning an objective function that is robust to these errors. The key idea of our approach is to learn a patch-based conditional discriminator which guides the texture optimization to be tolerant to misalignments. Our discriminator takes a synthesized view and a real image, and evaluates whether the synthesized one is realistic, under a broadened definition of realism. We train the discriminator by providing as `real' examples pairs of input views and their misaligned versions – so that the learned adversarial loss will tolerate errors from the scans. Experiments on synthetic and real data under quantitative or qualitative evaluation demonstrate the advantage of our approach in comparison to state of the art. Our code is publicly available with video demonstration.


page 1

page 3

page 5

page 6

page 8

page 12


Adaptive Joint Optimization for 3D Reconstruction with Differentiable Rendering

Due to inevitable noises introduced during scanning and quantization, 3D...

Deep Textured 3D Reconstruction of Human Bodies

Recovering textured 3D models of non-rigid human body shapes is challeng...

Texturify: Generating Textures on 3D Shape Surfaces

Texture cues on 3D objects are key to compelling visual representations,...

SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scans

We present SPSG, a novel approach to generate high-quality, colored 3D m...

Dynamic Object Removal and Spatio-Temporal RGB-D Inpainting via Geometry-Aware Adversarial Learning

Dynamic objects have a significant impact on the robot's perception of t...

Real-time texturing for 6D object instance detection from RGB Images

For objected detection, the availability of color cues strongly influenc...

Texture Synthesis with Recurrent Variational Auto-Encoder

We propose a recurrent variational auto-encoder for texture synthesis. A...

Please sign up or login with your details

Forgot password? Click here to reset