Generative Modelling With Inverse Heat Dissipation

by   Severi Rissanen, et al.

While diffusion models have shown great success in image generation, their noise-inverting generative process does not explicitly consider the structure of images, such as their inherent multi-scale nature. Inspired by diffusion models and the desirability of coarse-to-fine modelling, we propose a new model that generates images through iteratively inverting the heat equation, a PDE that locally erases fine-scale information when run over the 2D plane of the image. In our novel methodology, the solution of the forward heat equation is interpreted as a variational approximation in a directed graphical model. We demonstrate promising image quality and point out emergent qualitative properties not seen in diffusion models, such as disentanglement of overall colour and shape in images and aspects of neural network interpretability. Spectral analysis on natural images positions our model as a type of dual to diffusion models and reveals implicit inductive biases in them.


page 30

page 31

page 32

page 33

page 34

page 37

page 38

page 39


ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models

Denoising diffusion probabilistic models (DDPM) have shown remarkable pe...

An extended Krylov subspace method for decoding edge-based compressed images by homogeneous diffusion

The heat equation is often used in order to inpaint dropped data in inpa...

Dynamic Dual-Output Diffusion Models

Iterative denoising-based generation, also known as denoising diffusion ...

A Gray Level Indicator-Based Regularized Telegraph Diffusion Equation Applied to Image Despeckling

In this work, a gray level indicator based non-linear telegraph diffusio...

Schrödinger Diffusion for Shape Analysis with Texture

In recent years, quantities derived from the heat equation have become p...

DiffusionCLIP: Text-guided Image Manipulation Using Diffusion Models

Diffusion models are recent generative models that have shown great succ...

Multi-Scale Anisotropic Fourth-Order Diffusion Improves Ridge and Valley Localization

Ridge and valley enhancing filters are widely used in applications such ...