Post Selections Using Test Sets (PSUTS) and How Developmental Networks Avoid Them

06/19/2021
by   Juyang Weng, et al.
7

This paper raises a rarely reported practice in Artificial Intelligence (AI) called Post Selection Using Test Sets (PSUTS). Consequently, the popular error-backprop methodology in deep learning lacks an acceptable generalization power. All AI methods fall into two broad schools, connectionist and symbolic. The PSUTS fall into two kinds, machine PSUTS and human PSUTS. The connectionist school received criticisms for its "scruffiness" due to a huge number of network parameters and now the worse machine PSUTS; but the seemingly "clean" symbolic school seems more brittle because of a weaker generalization power using human PSUTS. This paper formally defines what PSUTS is, analyzes why error-backprop methods with random initial weights suffer from severe local minima, why PSUTS violates well-established research ethics, and how every paper that used PSUTS should have at least transparently reported PSUTS. For improved transparency in future publications, this paper proposes a new standard for performance evaluation of AI, called developmental errors for all networks trained, along with Three Learning Conditions: (1) an incremental learning architecture, (2) a training experience and (3) a limited amount of computational resources. Developmental Networks avoid PSUTS and are not "scruffy" because they drive Emergent Turing Machines and are optimal in the sense of maximum-likelihood across lifetime.

READ FULL TEXT

Authors

page 2

page 4

page 5

page 7

page 8

page 10

page 11

page 12

12/03/2015

A Study on Artificial Intelligence IQ and Standard Intelligent Model

Currently, potential threats of artificial intelligence (AI) to human ha...
09/02/2020

Problems in AI research and how the SP System may help to solve them

This paper describes problems in AI research and how the SP System may h...
06/09/2019

There is no general AI: Why Turing machines cannot pass the Turing test

Since 1950, when Alan Turing proposed what has since come to be called t...
12/19/2012

Towards common-sense reasoning via conditional simulation: legacies of Turing in Artificial Intelligence

The problem of replicating the flexibility of human common-sense reasoni...
08/07/2020

Uncontrollability of AI

Invention of artificial general intelligence is predicted to cause a shi...
11/23/2017

Improvised Comedy as a Turing Test

The best improvisational theatre actors can make any scene partner, of a...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.