Evolution of Novel Activation Functions in Neural Network Training with Applications to Classification of Exoplanets

06/01/2019
by   Snehanshu Saha, et al.
0

We present analytical exploration of novel activation functions as consequence of integration of several ideas leading to implementation and subsequent use in habitability classification of exoplanets. Neural networks, although a powerful engine in supervised methods, often require expensive tuning efforts for optimized performance. Habitability classes are hard to discriminate, especially when attributes used as hard markers of separation are removed from the data set. The solution is approached from the point of investigating analytical properties of the proposed activation functions. The theory of ordinary differential equations and fixed point are exploited to justify the "lack of tuning efforts" to achieve optimal performance compared to traditional activation functions. Additionally, the relationship between the proposed activation functions and the more popular ones is established through extensive analytical and empirical evidence. Finally, the activation functions have been implemented in plain vanilla feed-forward neural network to classify exoplanets.

READ FULL TEXT
research
02/24/2017

Activation Ensembles for Deep Neural Networks

Many activation functions have been proposed in the past, but selecting ...
research
04/14/2023

The R-mAtrIx Net

We provide a novel Neural Network architecture that can: i) output R-mat...
research
05/17/2021

How to Explain Neural Networks: A perspective of data space division

Interpretability of intelligent algorithms represented by deep learning ...
research
08/29/2022

Normalized Activation Function: Toward Better Convergence

Activation functions are essential for neural networks to introduce non-...
research
06/19/2023

Learn to Accumulate Evidence from All Training Samples: Theory and Practice

Evidential deep learning, built upon belief theory and subjective logic,...
research
08/18/2022

Lifted Bregman Training of Neural Networks

We introduce a novel mathematical formulation for the training of feed-f...
research
12/09/2022

Implementing Neural Network-Based Equalizers in a Coherent Optical Transmission System Using Field-Programmable Gate Arrays

In this work, we demonstrate the offline FPGA realization of both recurr...

Please sign up or login with your details

Forgot password? Click here to reset