A Constructive Approach for Data-Driven Randomized Learning of Feedforward Neural Networks

09/04/2019
by   Grzegorz Dudek, et al.
0

Feedforward neural networks with random hidden nodes suffer from a problem with the generation of random weights and biases as these are difficult to set optimally to obtain a good projection space. Typically, random parameters are drawn from an interval which is fixed before or adapted during the learning process. Due to the different functions of the weights and biases, selecting them both from the same interval is not a good idea. Recently more sophisticated methods of random parameters generation have been developed, such as the data-driven method proposed in Anon19, where the sigmoids are placed in randomly selected regions of the input space and then their slopes are adjusted to the local fluctuations of the target function. In this work, we propose an extended version of this method, which constructs iteratively the network architecture. This method successively generates new hidden nodes and accepts them if the training error decreases significantly. The threshold of acceptance is adapted to the current training stage. At the beginning of the training process only those nodes which lead to the largest error reduction are accepted. Then, the threshold is reduced by half to accept those nodes which model the target function details more accurately. This leads to faster convergence and more compact network architecture, as it includes only "significant" neurons. Several application examples are given which confirm this thesis.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/11/2019

Data-Driven Randomized Learning of Feedforward Neural Networks

Randomized methods of neural network learning suffer from a problem with...
research
08/15/2019

Improving Randomized Learning of Feedforward Neural Networks by Appropriate Generation of Random Parameters

In this work, a method of random parameters generation for randomized le...
research
10/13/2017

A Method of Generating Random Weights and Biases in Feedforward Neural Networks with Random Hidden Nodes

Neural networks with random hidden nodes have gained increasing interest...
research
07/04/2021

Autoencoder based Randomized Learning of Feedforward Neural Networks for Regression

Feedforward neural networks are widely used as universal predictive mode...
research
09/11/2021

An Insect-Inspired Randomly, Weighted Neural Network with Random Fourier Features For Neuro-Symbolic Relational Learning

Insects, such as fruit flies and honey bees, can solve simple associativ...
research
03/14/2023

Sequential three-way decisions with a single hidden layer feedforward neural network

The three-way decisions strategy has been employed to construct network ...
research
11/14/2020

AutoRWN: automatic construction and training of random weight networks using competitive swarm of agents

Random Weight Networks have been extensively used in many applications i...

Please sign up or login with your details

Forgot password? Click here to reset