Approximating Probability Distributions by ReLU Networks

01/25/2021
by   Manuj Mukherjee, et al.
0

How many neurons are needed to approximate a target probability distribution using a neural network with a given input distribution and approximation error? This paper examines this question for the case when the input distribution is uniform, and the target distribution belongs to the class of histogram distributions. We obtain a new upper bound on the number of required neurons, which is strictly better than previously existing upper bounds. The key ingredient in this improvement is an efficient construction of the neural nets representing piecewise linear functions. We also obtain a lower bound on the minimum number of neurons needed to approximate the histogram distributions.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

07/26/2021

High-Dimensional Distribution Generation Through Deep Neural Networks

We show that every d-dimensional probability distribution of bounded sup...
01/29/2021

On the capacity of deep generative networks for approximating distributions

We study the efficacy and efficiency of deep generative networks for app...
05/09/2018

Dispersion Bound for the Wyner-Ahlswede-Körner Network via Reverse Hypercontractivity on Types

Using the functional-entropic duality and the reverse hypercontractivity...
07/23/2019

Trainability and Data-dependent Initialization of Over-parameterized ReLU Neural Networks

A neural network is said to be over-specified if its representational po...
05/03/2015

Some Theoretical Properties of a Network of Discretely Firing Neurons

The problem of optimising a network of discretely firing neurons is addr...
09/14/2021

Histogram binning revisited with a focus on human perception

This paper presents a quantitative user study to evaluate how well users...
04/17/2018

Benford or not Benford: a systematic but not always well-founded use of an elegant law in experimental fields

In this paper, we will see that the proportion of d as leading digit, d ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.