An Analytic Layer-wise Deep Learning Framework with Applications to Robotics

02/07/2021
by   Huu-Thiet Nguyen, et al.
0

Deep learning has achieved great success in many applications, but it has been less well analyzed from the theoretical perspective. To deploy deep learning algorithms in a predictable and stable manner is particularly important in robotics, as robots are active agents that need to interact safely with the physical world. This paper presents an analytic deep learning framework for fully connected neural networks, which can be applied for both regression problems and classification problems. Examples for regression and classification problems include online robot control and robot vision. We present two layer-wise learning algorithms such that the convergence of the learning systems can be analyzed. Firstly, an inverse layer-wise learning algorithm for multilayer networks with convergence analysis for each layer is presented to understand the problems of layer-wise deep learning. Secondly, a forward progressive learning algorithm where the deep networks are built progressively by using single hidden layer networks is developed to achieve better accuracy. It is shown that the progressive learning method can be used for fine-tuning of weights from convergence point of view. The effectiveness of the proposed framework is illustrated based on classical benchmark recognition tasks using the MNIST and CIFAR-10 datasets and the results show a good balance between performance and explainability. The proposed method is subsequently applied for online learning of robot kinematics and experimental results on kinematic control of UR5e robot with unknown model are presented.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

02/14/2020

Layer-wise Pruning and Auto-tuning of Layer-wise Learning Rates in Fine-tuning of Deep Networks

Existing fine-tuning methods use a single learning rate over all layers....
11/16/2016

Cost-Sensitive Deep Learning with Layer-Wise Cost Estimation

While deep neural networks have succeeded in several visual applications...
03/06/2018

Deep Super Learner: A Deep Ensemble for Classification Problems

Deep learning has become very popular for tasks such as predictive model...
05/25/2021

Mixture of ELM based experts with trainable gating network

Mixture of experts method is a neural network based ensemble learning th...
09/30/2019

Full error analysis for the training of deep neural networks

Deep learning algorithms have been applied very successfully in recent y...
09/13/2014

A study on effectiveness of extreme learning machine

Extreme learning machine (ELM), proposed by Huang et al., has been shown...
05/22/2018

Parsimonious Bayesian deep networks

Combining Bayesian nonparametrics and a forward model selection strategy...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.