DiFT: Differentiable Differential Feature Transform for Multi-View Stereo

03/16/2022
by   Kaizhang Kang, et al.
0

We present a novel framework to automatically learn to transform the differential cues from a stack of images densely captured with a rotational motion into spatially discriminative and view-invariant per-pixel features at each view. These low-level features can be directly fed to any existing multi-view stereo technique for enhanced 3D reconstruction. The lighting condition during acquisition can also be jointly optimized in a differentiable fashion. We sample from a dozen of pre-scanned objects with a wide variety of geometry and reflectance to synthesize a large amount of high-quality training data. The effectiveness of our features is demonstrated on a number of challenging objects acquired with a lightstage, comparing favorably with state-of-the-art techniques. Finally, we explore additional applications of geometric detail visualization and computational stylization of complex appearance.

READ FULL TEXT

page 1

page 4

page 5

page 6

page 7

research
03/27/2021

Learning Efficient Photometric Feature Transform for Multi-view Stereo

We present a novel framework to learn to convert the perpixel photometri...
research
08/07/2023

Learning Photometric Feature Transform for Free-form Object Scan

We propose a novel framework to automatically learn to aggregate and tra...
research
03/29/2022

Efficient Reflectance Capture with a Deep Gated Mixture-of-Experts

We present a novel framework to efficiently acquire near-planar anisotro...
research
06/05/2023

Computational 3D topographic microscopy from terabytes of data per sample

We present a large-scale computational 3D topographic microscope that en...
research
04/02/2018

DeepMVS: Learning Multi-view Stereopsis

We present DeepMVS, a deep convolutional neural network (ConvNet) for mu...
research
11/24/2021

Extracting Triangular 3D Models, Materials, and Lighting From Images

We present an efficient method for joint optimization of topology, mater...
research
05/25/2017

Plan3D: Viewpoint and Trajectory Optimization for Aerial Multi-View Stereo Reconstruction

We introduce a new method that efficiently computes a set of rich viewpo...

Please sign up or login with your details

Forgot password? Click here to reset