Complex-valued Neural Networks with Non-parametric Activation Functions

02/22/2018
by   Simone Scardapane, et al.
0

Complex-valued neural networks (CVNNs) are a powerful modeling tool for domains where data can be naturally interpreted in terms of complex numbers. However, several analytical properties of the complex domain (e.g., holomorphicity) make the design of CVNNs a more challenging task than their real counterpart. In this paper, we consider the problem of flexible activation functions (AFs) in the complex domain, i.e., AFs endowed with sufficient degrees of freedom to adapt their shape given the training data. While this problem has received considerable attention in the real case, a very limited literature exists for CVNNs, where most activation functions are generally developed in a split fashion (i.e., by considering the real and imaginary parts of the activation separately) or with simple phase-amplitude techniques. Leveraging over the recently proposed kernel activation functions (KAFs), and related advances in the design of complex-valued kernels, we propose the first fully complex, non-parametric activation function for CVNNs, which is based on a kernel expansion with a fixed dictionary that can be implemented efficiently on vectorized hardware. Several experiments on common use cases, including prediction and channel equalization, validate our proposal when compared to real-valued neural networks and CVNNs with fixed activation functions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/06/2019

Widely Linear Kernels for Complex-Valued Kernel Activation Functions

Complex-valued neural networks (CVNNs) have been shown to be powerful no...
research
07/13/2017

Kafnets: kernel-based non-parametric activation functions for neural networks

Neural networks are generally built by interleaving (adaptable) linear l...
research
02/26/2018

Improving Graph Convolutional Networks with Non-Parametric Activation Functions

Graph neural networks (GNNs) are a class of neural networks that allow t...
research
01/29/2019

Multikernel activation functions: formulation and a case study

The design of activation functions is a growing research area in the fie...
research
02/14/2019

A Broad Class of Discrete-Time Hypercomplex-Valued Hopfield Neural Networks

In this paper, we address the stability of a broad class of discrete-tim...
research
02/16/2023

Theory and Implementation of Complex-Valued Neural Networks

This work explains in detail the theory behind Complex-Valued Neural Net...
research
03/28/2019

On the Stability and Generalization of Learning with Kernel Activation Functions

In this brief we investigate the generalization properties of a recently...

Please sign up or login with your details

Forgot password? Click here to reset