DeepAI
Log In Sign Up

Universal Approximation Theorems of Fully Connected Binarized Neural Networks

02/04/2021
by   Mikail Yayla, et al.
0

Neural networks (NNs) are known for their high predictive accuracy in complex learning problems. Beside practical advantages, NNs also indicate favourable theoretical properties such as universal approximation (UA) theorems. Binarized Neural Networks (BNNs) significantly reduce time and memory demands by restricting the weight and activation domains to two values. Despite the practical advantages, theoretical guarantees based on UA theorems of BNNs are rather sparse in the literature. We close this gap by providing UA theorems for fully connected BNNs under the following scenarios: (1) for binarized inputs, UA can be constructively achieved under one hidden layer; (2) for inputs with real numbers, UA can not be achieved under one hidden layer but can be constructively achieved under two hidden layers for Lipschitz-continuous functions. Our results indicate that fully connected BNNs can approximate functions universally, under certain conditions.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/29/2019

On the rate of convergence of fully connected very deep neural network regression estimates

Recent results in nonparametric regression show that deep learning, i.e....
10/21/2019

Approximation capabilities of neural networks on unbounded domains

We prove universal approximation theorems of neural networks in L^p(R× [...
07/30/2021

Pruning Neural Networks with Interpolative Decompositions

We introduce a principled approach to neural network pruning that casts ...
06/23/2021

Universal Consistency of Deep Convolutional Neural Networks

Compared with avid research activities of deep convolutional neural netw...
10/26/2020

The estimation of training accuracy for two-layer neural networks on random datasets without training

Although the neural network (NN) technique plays an important role in ma...
07/05/2022

Correlation between entropy and generalizability in a neural network

Although neural networks can solve very complex machine-learning problem...
06/08/2021

Householder-Absolute Neural Layers For High Variability and Deep Trainability

We propose a new architecture for artificial neural networks called Hous...