Score Matching Model for Unbounded Data Score

06/10/2021
by   Dongjun Kim, et al.
12

Recent advance in score-based models incorporates the stochastic differential equation (SDE), which brings the state-of-the art performance on image generation tasks. This paper improves such score-based models by analyzing the model at the zero perturbation noise. In real datasets, the score function diverges as the perturbation noise (σ) decreases to zero, and this observation leads an argument that the score estimation fails at σ=0 with any neural network structure. Subsequently, we introduce Unbounded Noise Conditional Score Network (UNCSN) that resolves the score diverging problem with an easily applicable modification to any noise conditional score-based models. Additionally, we introduce a new type of SDE, so the exact log likelihood can be calculated from the newly suggested SDE. On top of that, the associated loss function mitigates the loss imbalance issue in a mini-batch, and we present a theoretic analysis on the proposed loss to uncover the behind mechanism of the data distribution modeling by the score-based models.

READ FULL TEXT

page 9

page 18

page 19

page 20

page 21

page 22

page 23

page 24

research
07/12/2023

Energy Discrepancies: A Score-Independent Loss for Energy-Based Models

Energy-based models are a simple yet powerful class of probabilistic mod...
research
10/17/2019

Annealed Denoising Score Matching: Learning Energy-Based Models in High-Dimensional Spaces

Energy-Based Models (EBMs) outputs unmormalized log-probability values g...
research
10/10/2022

Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Models

We introduce Sequential Neural Posterior Score Estimation (SNPSE) and Se...
research
10/09/2022

Regularizing Score-based Models with Score Fokker-Planck Equations

Score-based generative models learn a family of noise-conditional score ...
research
06/13/2023

Learning Unnormalized Statistical Models via Compositional Optimization

Learning unnormalized statistical models (e.g., energy-based models) is ...
research
03/27/2022

Denoising Likelihood Score Matching for Conditional Score-based Data Generation

Many existing conditional score-based data generation methods utilize Ba...
research
12/21/2020

Nonstationarity Analysis of Materials Microstructures via Fisher Score Vectors

Microstructures are critical to the physical properties of materials. St...

Please sign up or login with your details

Forgot password? Click here to reset