Learnability and Robustness of Shallow Neural Networks Learned With a Performance-Driven BP and a Variant PSO For Edge Decision-Making

08/13/2020
by   Hongmei He, et al.
0

In many cases, the computing resources are limited without the benefit from GPU, especially in the edge devices of IoT enabled systems. It may not be easy to implement complex AI models in edge devices. The Universal Approximation Theorem states that a shallow neural network (SNN) can represent any nonlinear function. However, how fat is an SNN enough to solve a nonlinear decision-making problem in edge devices? In this paper, we focus on the learnability and robustness of SNNs, obtained by a greedy tight force heuristic algorithm (performance driven BP) and a loose force meta-heuristic algorithm (a variant of PSO). Two groups of experiments are conducted to examine the learnability and the robustness of SNNs with Sigmoid activation, learned/optimised by KPI-PDBPs and KPI-VPSOs, where, KPIs (key performance indicators: error (ERR), accuracy (ACC) and F_1 score) are the objectives, driving the searching process. An incremental approach is applied to examine the impact of hidden neuron numbers on the performance of SNNs, learned/optimised by KPI-PDBPs and KPI-VPSOs. From the engineering prospective, all sensors are well justified for a specific task. Hence, all sensor readings should be strongly correlated to the target. Therefore, the structure of an SNN should depend on the dimensions of a problem space. The experimental results show that the number of hidden neurons up to the dimension number of a problem space is enough; the learnability of SNNs, produced by KPI-PDBP, is better than that of SNNs, optimized by KPI-VPSO, regarding the performance and learning time on the training data sets; the robustness of SNNs learned by KPI-PDBPs and KPI-VPSOs depends on the data sets; and comparing with other classic machine learning models, ACC-PDBPs win for almost all tested data sets.

READ FULL TEXT

page 27

page 28

research
04/02/2020

Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations

The need for more transparency of the decision-making processes in artif...
research
04/21/2014

Influence of the learning method in the performance of feedforward neural networks when the activity of neurons is modified

A method that allows us to give a different treatment to any neuron insi...
research
09/06/2018

Applying Deep Learning to Derivatives Valuation

The universal approximation theorem of artificial neural networks states...
research
02/06/2023

Using Learned Indexes to Improve Time Series Indexing Performance on Embedded Sensor Devices

Efficiently querying data on embedded sensor and IoT devices is challeng...
research
06/08/2020

A Heuristically Self-Organised Linguistic Attribute Deep Learning in Edge Computing For IoT Intelligence

With the development of Internet of Things (IoT), IoT intelligence becom...
research
07/29/2020

On the Use of Interpretable Machine Learning for the Management of Data Quality

Data quality is a significant issue for any application that requests fo...

Please sign up or login with your details

Forgot password? Click here to reset