Post Selections Using Test Sets (PSUTS) and How Developmental Networks Avoid Them

by   Juyang Weng, et al.

This paper raises a rarely reported practice in Artificial Intelligence (AI) called Post Selection Using Test Sets (PSUTS). Consequently, the popular error-backprop methodology in deep learning lacks an acceptable generalization power. All AI methods fall into two broad schools, connectionist and symbolic. The PSUTS fall into two kinds, machine PSUTS and human PSUTS. The connectionist school received criticisms for its "scruffiness" due to a huge number of network parameters and now the worse machine PSUTS; but the seemingly "clean" symbolic school seems more brittle because of a weaker generalization power using human PSUTS. This paper formally defines what PSUTS is, analyzes why error-backprop methods with random initial weights suffer from severe local minima, why PSUTS violates well-established research ethics, and how every paper that used PSUTS should have at least transparently reported PSUTS. For improved transparency in future publications, this paper proposes a new standard for performance evaluation of AI, called developmental errors for all networks trained, along with Three Learning Conditions: (1) an incremental learning architecture, (2) a training experience and (3) a limited amount of computational resources. Developmental Networks avoid PSUTS and are not "scruffy" because they drive Emergent Turing Machines and are optimal in the sense of maximum-likelihood across lifetime.


page 2

page 4

page 5

page 7

page 8

page 10

page 11

page 12


A Study on Artificial Intelligence IQ and Standard Intelligent Model

Currently, potential threats of artificial intelligence (AI) to human ha...

Problems in AI research and how the SP System may help to solve them

This paper describes problems in AI research and how the SP System may h...

Ethical Artificial Intelligence

This book-length article combines several peer reviewed papers and new m...

Towards common-sense reasoning via conditional simulation: legacies of Turing in Artificial Intelligence

The problem of replicating the flexibility of human common-sense reasoni...

On Controllability of AI

Invention of artificial general intelligence is predicted to cause a shi...

Improvised Comedy as a Turing Test

The best improvisational theatre actors can make any scene partner, of a...

Please sign up or login with your details

Forgot password? Click here to reset