GramGAN: Deep 3D Texture Synthesis From 2D Exemplars

06/29/2020
by   Tiziano Portenier, et al.
0

We present a novel texture synthesis framework, enabling the generation of infinite, high-quality 3D textures given a 2D exemplar image. Inspired by recent advances in natural texture synthesis, we train deep neural models to generate textures by non-linearly combining learned noise frequencies. To achieve a highly realistic output conditioned on an exemplar patch, we propose a novel loss function that combines ideas from both style transfer and generative adversarial networks. In particular, we train the synthesis network to match the Gram matrices of deep features from a discriminator network. In addition, we propose two architectural concepts and an extrapolation strategy that significantly improve generalization performance. In particular, we inject both model input and condition into hidden network layers by learning to scale and bias hidden activations. Quantitative and qualitative evaluations on a diverse set of exemplars motivate our design decisions and show that our system performs superior to previous state of the art. Finally, we conduct a user study that confirms the benefits of our framework.

READ FULL TEXT

page 7

page 12

page 13

page 14

page 16

page 17

page 18

page 19

research
06/09/2017

TextureGAN: Controlling Deep Image Synthesis with Texture Patches

In this paper, we investigate deep image synthesis guided by sketch, col...
research
05/18/2017

Learning Texture Manifolds with the Periodic Spatial GAN

This paper introduces a novel approach to texture synthesis based on gen...
research
04/15/2016

Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks

This paper proposes Markovian Generative Adversarial Networks (MGANs), a...
research
01/13/2022

SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Maps

We present SeamlessGAN, a method capable of automatically generating til...
research
12/23/2017

Texture Synthesis with Recurrent Variational Auto-Encoder

We propose a recurrent variational auto-encoder for texture synthesis. A...
research
01/29/2019

Applying Visual Domain Style Transfer and Texture Synthesis Techniques to Audio - Insights and Challenges

Style transfer is a technique for combining two images based on the acti...
research
06/30/2020

Deep Geometric Texture Synthesis

Recently, deep generative adversarial networks for image generation have...

Please sign up or login with your details

Forgot password? Click here to reset