Approximating Probability Distributions by ReLU Networks

by   Manuj Mukherjee, et al.

How many neurons are needed to approximate a target probability distribution using a neural network with a given input distribution and approximation error? This paper examines this question for the case when the input distribution is uniform, and the target distribution belongs to the class of histogram distributions. We obtain a new upper bound on the number of required neurons, which is strictly better than previously existing upper bounds. The key ingredient in this improvement is an efficient construction of the neural nets representing piecewise linear functions. We also obtain a lower bound on the minimum number of neurons needed to approximate the histogram distributions.



There are no comments yet.


page 1

page 2

page 3

page 4


High-Dimensional Distribution Generation Through Deep Neural Networks

We show that every d-dimensional probability distribution of bounded sup...

On the capacity of deep generative networks for approximating distributions

We study the efficacy and efficiency of deep generative networks for app...

Dispersion Bound for the Wyner-Ahlswede-Körner Network via Reverse Hypercontractivity on Types

Using the functional-entropic duality and the reverse hypercontractivity...

Trainability and Data-dependent Initialization of Over-parameterized ReLU Neural Networks

A neural network is said to be over-specified if its representational po...

Some Theoretical Properties of a Network of Discretely Firing Neurons

The problem of optimising a network of discretely firing neurons is addr...

Histogram binning revisited with a focus on human perception

This paper presents a quantitative user study to evaluate how well users...

Benford or not Benford: a systematic but not always well-founded use of an elegant law in experimental fields

In this paper, we will see that the proportion of d as leading digit, d ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.