Log In Sign Up

A Theory on AI Uncertainty Based on Rademacher Complexity and Shannon Entropy

by   Mingyong Zhou, et al.

In this paper, we present a theoretical discussion on AI deep learning neural network uncertainty investigation based on the classical Rademacher complexity and Shannon entropy. First it is shown that the classical Rademacher complexity and Shannon entropy is closely related by quantity by definitions. Secondly based on the Shannon mathematical theory on communication [3], we derive a criteria to ensure AI correctness and accuracy in classifications problems. Last but not the least based on Peter Barlette's work, we show both a relaxing condition and a stricter condition to guarantee the correctness and accuracy in AI classification . By elucidating in this paper criteria condition in terms of Shannon entropy based on Shannon theory, it becomes easier to explore other criteria in terms of other complexity measurements such as Vapnik-Cheronenkis, Gaussian complexity by taking advantage of the relations studies results in other references. A close to 0.5 criteria on Shannon entropy is derived in this paper for the theoretical investigation of AI accuracy and correctness for classification problems.


page 1

page 2

page 3

page 4


AI Uncertainty Based on Rademacher Complexity and Shannon Entropy

In this paper from communication channel coding perspective we are able ...

Fuzzy and entropy facial recognition

This paper suggests an effective method for facial recognition using fuz...

A genuinely natural information measure

The theoretical measuring of information was famously initiated by Shann...

The Production of Probabilistic Entropy in Structure/Action Contingency Relations

Luhmann (1984) defined society as a communication system which is struct...

Deep learning using Havrda-Charvat entropy for classification of pulmonary endomicroscopy

Pulmonary optical endomicroscopy (POE) is an imaging technology in real ...

A Bayesian Monte-Carlo Uncertainty Model for Assessment of Shear Stress Entropy

The entropy models have been recently adopted in many studies to evaluat...

On regularized Shannon sampling formulas with localized sampling

In this paper we present new regularized Shannon sampling formulas which...