Stacked Autoencoder Based Deep Random Vector Functional Link Neural Network for Classification

by   Rakesh Katuwal, et al.

Extreme learning machine (ELM), which can be viewed as a variant of Random Vector Functional Link (RVFL) network without the input-output direct connections, has been extensively used to create multi-layer (deep) neural networks. Such networks employ randomization based autoencoders (AE) for unsupervised feature extraction followed by an ELM classifier for final decision making. Each randomization based AE acts as an independent feature extractor and a deep network is obtained by stacking several such AEs. Inspired by the better performance of RVFL over ELM, in this paper, we propose several deep RVFL variants by utilizing the framework of stacked autoencoders. Specifically, we introduce direct connections (feature reuse) from preceding layers to the fore layers of the network as in the original RVFL network. Such connections help to regularize the randomization and also reduce the model complexity. Furthermore, we also introduce denoising criterion, recovering clean inputs from their corrupted versions, in the autoencoders to achieve better higher level representations than the ordinary autoencoders. Extensive experiments on several classification datasets show that our proposed deep networks achieve overall better and faster generalization than the other relevant state-of-the-art deep neural networks.


page 21

page 22


Random Vector Functional Link Neural Network based Ensemble Deep Learning

In this paper, we propose a deep learning framework based on randomized ...

How to Train Your Deep Neural Network with Dictionary Learning

Currently there are two predominant ways to train deep neural networks. ...

Genesis of Basic and Multi-Layer Echo State Network Recurrent Autoencoders for Efficient Data Representations

It is a widely accepted fact that data representations intervene noticea...

Weighting and Pruning based Ensemble Deep Random Vector Functional Link Network for Tabular Data Classification

In this paper, we first introduce batch normalization to the edRVFL netw...

A Peek Into the Hidden Layers of a Convolutional Neural Network Through a Factorization Lens

Despite their increasing popularity and success in a variety of supervis...

Random vector functional link network: recent developments, applications, and future directions

Neural networks have been successfully employed in various domain such a...

Integration of Autoencoder and Functional Link Artificial Neural Network for Multi-label Classification

Multi-label (ML) classification is an actively researched topic currentl...