An Analytic Layer-wise Deep Learning Framework with Applications to Robotics

02/07/2021
by   Huu-Thiet Nguyen, et al.
0

Deep learning has achieved great success in many applications, but it has been less well analyzed from the theoretical perspective. To deploy deep learning algorithms in a predictable and stable manner is particularly important in robotics, as robots are active agents that need to interact safely with the physical world. This paper presents an analytic deep learning framework for fully connected neural networks, which can be applied for both regression problems and classification problems. Examples for regression and classification problems include online robot control and robot vision. We present two layer-wise learning algorithms such that the convergence of the learning systems can be analyzed. Firstly, an inverse layer-wise learning algorithm for multilayer networks with convergence analysis for each layer is presented to understand the problems of layer-wise deep learning. Secondly, a forward progressive learning algorithm where the deep networks are built progressively by using single hidden layer networks is developed to achieve better accuracy. It is shown that the progressive learning method can be used for fine-tuning of weights from convergence point of view. The effectiveness of the proposed framework is illustrated based on classical benchmark recognition tasks using the MNIST and CIFAR-10 datasets and the results show a good balance between performance and explainability. The proposed method is subsequently applied for online learning of robot kinematics and experimental results on kinematic control of UR5e robot with unknown model are presented.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/14/2020

Layer-wise Pruning and Auto-tuning of Layer-wise Learning Rates in Fine-tuning of Deep Networks

Existing fine-tuning methods use a single learning rate over all layers....
research
11/16/2016

Cost-Sensitive Deep Learning with Layer-Wise Cost Estimation

While deep neural networks have succeeded in several visual applications...
research
03/06/2018

Deep Super Learner: A Deep Ensemble for Classification Problems

Deep learning has become very popular for tasks such as predictive model...
research
05/25/2021

Mixture of ELM based experts with trainable gating network

Mixture of experts method is a neural network based ensemble learning th...
research
09/30/2019

Full error analysis for the training of deep neural networks

Deep learning algorithms have been applied very successfully in recent y...
research
09/13/2014

A study on effectiveness of extreme learning machine

Extreme learning machine (ELM), proposed by Huang et al., has been shown...
research
05/22/2018

Parsimonious Bayesian deep networks

Combining Bayesian nonparametrics and a forward model selection strategy...

Please sign up or login with your details

Forgot password? Click here to reset