SinDiffusion: Learning a Diffusion Model from a Single Natural Image

11/22/2022
by   Weilun Wang, et al.
0

We present SinDiffusion, leveraging denoising diffusion models to capture internal distribution of patches from a single natural image. SinDiffusion significantly improves the quality and diversity of generated samples compared with existing GAN-based approaches. It is based on two core designs. First, SinDiffusion is trained with a single model at a single scale instead of multiple models with progressive growing of scales which serves as the default setting in prior work. This avoids the accumulation of errors, which cause characteristic artifacts in generated results. Second, we identify that a patch-level receptive field of the diffusion network is crucial and effective for capturing the image's patch statistics, therefore we redesign the network structure of the diffusion model. Coupling these two designs enables us to generate photorealistic and diverse images from a single image. Furthermore, SinDiffusion can be applied to various applications, i.e., text-guided image generation, and image outpainting, due to the inherent capability of diffusion models. Extensive experiments on a wide range of images demonstrate the superiority of our proposed method for modeling the patch distribution.

READ FULL TEXT

page 1

page 5

page 6

page 7

page 12

page 13

page 14

page 15

research
11/29/2022

SinDDM: A Single Image Denoising Diffusion Model

Denoising diffusion models (DDMs) have led to staggering performance lea...
research
05/02/2019

SinGAN: Learning a Generative Model from a Single Natural Image

We introduce SinGAN, an unconditional generative model that can be learn...
research
03/03/2022

PetsGAN: Rethinking Priors for Single Image Generation

Single image generation (SIG), described as generating diverse samples t...
research
03/27/2023

Diffusion Models for Memory-efficient Processing of 3D Medical Images

Denoising diffusion models have recently achieved state-of-the-art perfo...
research
04/25/2023

Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models

Diffusion models are powerful, but they require a lot of time and data t...
research
06/22/2020

Hierarchical Patch VAE-GAN: Generating Diverse Videos from a Single Sample

We consider the task of generating diverse and novel videos from a singl...
research
05/24/2023

Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shape

Synthesizing novel 3D models that resemble the input example has long be...

Please sign up or login with your details

Forgot password? Click here to reset