Neural Photometry-guided Visual Attribute Transfer

We present a deep learning-based method for propagating spatially-varying visual material attributes (e.g. texture maps or image stylizations) to larger samples of the same or similar materials. For training, we leverage images of the material taken under multiple illuminations and a dedicated data augmentation policy, making the transfer robust to novel illumination conditions and affine deformations. Our model relies on a supervised image-to-image translation framework and is agnostic to the transferred domain; we showcase a semantic segmentation, a normal map, and a stylization. Following an image analogies approach, the method only requires the training data to contain the same visual structures as the input guidance. Our approach works at interactive rates, making it suitable for material edit applications. We thoroughly evaluate our learning methodology in a controlled setup providing quantitative measures of performance. Last, we demonstrate that training the model on a single material is enough to generalize to materials of the same type without the need for massive datasets.

READ FULL TEXT

page 1

page 5

page 6

page 7

page 8

page 9

page 10

page 11

research
05/09/2022

Photo-to-Shape Material Transfer for Diverse Structures

We introduce a method for assigning photorealistic relightable materials...
research
07/03/2023

NeuBTF: Neural fields for BTF encoding and transfer

Neural material representations are becoming a popular way to represent ...
research
05/22/2023

Materialistic: Selecting Similar Materials in Images

Separating an image into meaningful underlying components is a crucial f...
research
09/23/2019

Shadow Transfer: Single Image Relighting For Urban Road Scenes

Illumination effects in images, specifically cast shadows and shading, h...
research
01/03/2018

Live Intrinsic Material Estimation

We present the first end-to-end approach for real-time material estimati...
research
08/20/2018

Image-based remapping of spatially-varying material appearance

BRDF models are ubiquitous tools for the representation of material appe...
research
09/04/2018

Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networks

We present a convolutional neural network (CNN) based solution for model...

Please sign up or login with your details

Forgot password? Click here to reset