The universal approximation theorem for complex-valued neural networks

12/06/2020
by   Felix Voigtlaender, et al.
0

We generalize the classical universal approximation theorem for neural networks to the case of complex-valued neural networks. Precisely, we consider feedforward networks with a complex activation function σ : ℂ→ℂ in which each neuron performs the operation ℂ^N →ℂ, z ↦σ(b + w^T z) with weights w ∈ℂ^N and a bias b ∈ℂ, and with σ applied componentwise. We completely characterize those activation functions σ for which the associated complex networks have the universal approximation property, meaning that they can uniformly approximate any continuous function on any compact subset of ℂ^d arbitrarily well. Unlike the classical case of real networks, the set of "good activation functions" which give rise to networks with the universal approximation property differs significantly depending on whether one considers deep networks or shallow networks: For deep networks with at least two hidden layers, the universal approximation property holds as long as σ is neither a polynomial, a holomorphic function, or an antiholomorphic function. Shallow networks, on the other hand, are universal if and only if the real part or the imaginary part of σ is not a polyharmonic function.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/26/2023

Universal approximation with complex-valued deep narrow neural networks

We study the universality of complex-valued neural networks with bounded...
research
02/16/2020

A closer look at the approximation capabilities of neural networks

The universal approximation theorem, in one of its most general versions...
research
07/12/2020

Universal Approximation Power of Deep Neural Networks via Nonlinear Control Theory

In this paper, we explain the universal approximation capabilities of de...
research
01/27/2021

Kähler Geometry of Quiver Varieties and Machine Learning

We develop an algebro-geometric formulation for neural networks in machi...
research
10/28/2020

Estimating Multiplicative Relations in Neural Networks

Universal approximation theorem suggests that a shallow neural network c...
research
03/25/2022

Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activation

In this article, we prove approximation theorems in classes of deep and ...
research
06/28/2018

ResNet with one-neuron hidden layers is a Universal Approximator

We demonstrate that a very deep ResNet with stacked modules with one neu...

Please sign up or login with your details

Forgot password? Click here to reset