Few-Shot Diffusion Models

05/30/2022
by   Giorgio Giannone, et al.
3

Denoising diffusion probabilistic models (DDPM) are powerful hierarchical latent variable models with remarkable sample generation quality and training stability. These properties can be attributed to parameter sharing in the generative hierarchy, as well as a parameter-free diffusion-based inference procedure. In this paper, we present Few-Shot Diffusion Models (FSDM), a framework for few-shot generation leveraging conditional DDPMs. FSDMs are trained to adapt the generative process conditioned on a small set of images from a given class by aggregating image patch information using a set-based Vision Transformer (ViT). At test time, the model is able to generate samples from previously unseen classes conditioned on as few as 5 samples from that class. We empirically show that FSDM can perform few-shot generation and transfer to new datasets. We benchmark variants of our method on complex vision datasets for few-shot learning and compare to unconditional and conditional DDPM baselines. Additionally, we show how conditioning the model on patch-based input set information improves training convergence.

READ FULL TEXT

page 1

page 8

page 17

page 18

page 19

page 20

page 21

research
06/02/2023

Conditional Generation from Unconditional Diffusion Models using Denoiser Representations

Denoising diffusion models have gained popularity as a generative modeli...
research
04/25/2023

Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models

Diffusion models are powerful, but they require a lot of time and data t...
research
10/23/2021

Hierarchical Few-Shot Generative Models

A few-shot generative model should be able to generate data from a distr...
research
06/15/2022

Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models

Diffusion probabilistic models (DPMs) are a class of powerful deep gener...
research
03/15/2023

Generating symbolic music using diffusion models

Probabilistic Denoising Diffusion models have emerged as simple yet very...
research
06/15/2022

Rethinking Generalization in Few-Shot Classification

Single image-level annotations only correctly describe an often small su...
research
02/21/2018

A Generative Deep Recurrent Model for Exchangeable Data

We present a novel model architecture which leverages deep learning tool...

Please sign up or login with your details

Forgot password? Click here to reset