A likelihood approach to nonparametric estimation of a singular distribution using deep generative models

05/09/2021
by   Minwoo Chae, et al.
8

We investigate statistical properties of a likelihood approach to nonparametric estimation of a singular distribution using deep generative models. More specifically, a deep generative model is used to model high-dimensional data that are assumed to concentrate around some low-dimensional structure. Estimating the distribution supported on this low-dimensional structure such as a low-dimensional manifold is challenging due to its singularity with respect to the Lebesgue measure in the ambient space. In the considered model, a usual likelihood approach can fail to estimate the target distribution consistently due to the singularity. We prove that a novel and effective solution exists by perturbing the data with an instance noise which leads to consistent estimation of the underlying distribution with desirable convergence rates. We also characterize the class of distributions that can be efficiently estimated via deep generative models. This class is sufficiently general to contain various structured distributions such as product distributions, classically smooth distributions and distributions supported on a low-dimensional manifold. Our analysis provides some insights on how deep generative models can avoid the curse of dimensionality for nonparametric distribution estimation. We conduct thorough simulation study and real data analysis to empirically demonstrate that the proposed data perturbation technique improves the estimation performance significantly.

READ FULL TEXT

page 20

page 21

page 23

research
02/07/2022

Rates of convergence for nonparametric estimation of singular distributions using generative adversarial networks

We consider generative adversarial networks (GAN) for estimating paramet...
research
04/14/2022

Diagnosing and Fixing Manifold Overfitting in Deep Generative Models

Likelihood-based, or explicit, deep generative models use neural network...
research
05/31/2018

Ratio Matching MMD Nets: Low dimensional projections for effective deep generative models

Deep generative models can learn to generate realistic-looking images on...
research
11/30/2022

Denoising Deep Generative Models

Likelihood-based deep generative models have recently been shown to exhi...
research
06/16/2023

Understanding Deep Generative Models with Generalized Empirical Likelihoods

Understanding how well a deep generative model captures a distribution o...
research
06/30/2021

On the Generative Utility of Cyclic Conditionals

We study whether and how can we model a joint distribution p(x,z) using ...
research
06/08/2021

Manifold Topology Divergence: a Framework for Comparing Data Manifolds

We develop a framework for comparing data manifolds, aimed, in particula...

Please sign up or login with your details

Forgot password? Click here to reset