Minimax Rate of Distribution Estimation on Unknown Submanifold under Adversarial Losses

02/18/2022
by   Rong Tang, et al.
0

Statistical inference from high-dimensional data with low-dimensional structures has recently attracted lots of attention. In machine learning, deep generative modeling approaches implicitly estimate distributions of complex objects by creating new samples from the underlying distribution, and have achieved great success in generating synthetic realistic-looking images and texts. A key step in these approaches is the extraction of latent features or representations (encoding) that can be used for accurately reconstructing the original data (decoding). In other words, low-dimensional manifold structure is implicitly assumed and utilized in the distribution modeling and estimation. To understand the benefit of low-dimensional manifold structure in generative modeling, we build a general minimax framework for distribution estimation on unknown submanifold under adversarial losses, with suitable smoothness assumptions on the target distribution and the manifold. The established minimax rate elucidates how various problem characteristics, including intrinsic dimensionality of the data and smoothness levels of the target distribution and the manifold, affect the fundamental limit of high-dimensional distribution estimation. To prove the minimax upper bound, we construct an estimator based on a mixture of locally fitted generative models, which is motivated by the partition of unity technique from differential geometry and is necessary to cover cases where the underlying data manifold does not admit a global parametrization. We also propose a data-driven adaptive estimator that is shown to simultaneously attain within a logarithmic factor of the optimal rate over a large collection of distribution classes.

READ FULL TEXT
research
02/25/2023

On Deep Generative Models for Approximation and Estimation of Distributions on Manifolds

Generative networks have experienced great empirical successes in distri...
research
01/02/2023

Estimating Distributions with Low-dimensional Structures Using Mixtures of Generative Models

There has been a growing interest in statistical inference from data sat...
research
02/08/2019

Generating the support with extreme value losses

When optimizing against the mean loss over a distribution of predictions...
research
02/07/2022

Rates of convergence for nonparametric estimation of singular distributions using generative adversarial networks

We consider generative adversarial networks (GAN) for estimating paramet...
research
06/29/2014

Estimating the distribution of Galaxy Morphologies on a continuous space

The incredible variety of galaxy shapes cannot be summarized by human de...
research
04/14/2022

Diagnosing and Fixing Manifold Overfitting in Deep Generative Models

Likelihood-based, or explicit, deep generative models use neural network...
research
05/26/2018

Geometric Understanding of Deep Learning

Deep learning is the mainstream technique for many machine learning task...

Please sign up or login with your details

Forgot password? Click here to reset