Operational Neural Networks

02/15/2019
by   Serkan Kiranyaz, et al.
0

Feed-forward, fully-connected Artificial Neural Networks (ANNs) or the so-called Multi-Layer Perceptrons (MLPs) are well-known universal approximators. However, their learning performance varies significantly depending on the function or the solution space that they attempt to approximate. This is mainly because of their homogenous configuration based solely on the linear neuron model. Therefore, while they learn very well those problems with a monotonous, relatively simple and linearly separable solution space, they may entirely fail to do so when the solution space is highly nonlinear and complex. Sharing the same linear neuron model with two additional constraints (local connections and weight sharing), this is also true for the conventional Convolutional Neural Networks (CNNs) and, it is, therefore, not surprising that in many challenging problems only the deep CNNs with a massive complexity and depth can achieve the required diversity and the learning performance. In order to address this drawback and also to accomplish a more generalized model over the convolutional neurons, this study proposes a novel network model, called Operational Neural Networks (ONNs), which can be heterogeneous and encapsulate neurons with any set of operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. Finally, a novel training method is formulated to back-propagate the error through the operational layers of ONNs. Experimental results over highly challenging problems demonstrate the superior learning capabilities of ONNs even with few neurons and hidden layers.

READ FULL TEXT

page 5

page 7

page 8

page 9

research
08/21/2020

Exploiting Heterogeneity in Operational Neural Networks by Synaptic Plasticity

The recently proposed network model, Operational Neural Networks (ONNs),...
research
04/24/2020

Self-Organized Operational Neural Networks with Generative Neurons

Operational Neural Networks (ONNs) have recently been proposed to addres...
research
09/01/2020

Operational vs Convolutional Neural Networks for Image Denoising

Convolutional Neural Networks (CNNs) have recently become a favored tech...
research
07/17/2022

2D Self-Organized ONN Model For Handwritten Text Recognition

Deep Convolutional Neural Networks (CNNs) have recently reached state-of...
research
06/03/2020

FastONN – Python based open-source GPU implementation for Operational Neural Networks

Operational Neural Networks (ONNs) have recently been proposed as a spec...
research
08/04/2022

Modular Grammatical Evolution for the Generation of Artificial Neural Networks

This paper presents a novel method, called Modular Grammatical Evolution...
research
09/14/2023

A Multi-In and Multi-Out Dendritic Neuron Model and its Optimization

Artificial neural networks (ANNs), inspired by the interconnection of re...

Please sign up or login with your details

Forgot password? Click here to reset