DeepAI AI Chat
Log In Sign Up

Testing System Intelligence

by   Joseph Sifakis, et al.

We discuss the adequacy of tests for intelligent systems and practical problems raised by their implementation. We propose the replacement test as the ability of a system to replace successfully another system performing a task in a given context. We show how it can characterize salient aspects of human intelligence that cannot be taken into account by the Turing test. We argue that building intelligent systems passing the replacement test involves a series of technical problems that are outside the scope of current AI. We present a framework for implementing the proposed test and validating the properties of the intelligent systems. We discuss the inherent limitations of intelligent system validation and advocate new theoretical foundations for extending existing rigorous test methods. We suggest that the replacement test, based on the complementarity of skills between human and machine, can lead to a multitude of intelligence concepts reflecting the ability to combine data-based and symbolic knowledge to varying degrees.


page 6

page 13


The Inescapable Duality of Data and Knowledge

We will discuss how over the last 30 to 50 years, systems that focused o...

The Lovelace 2.0 Test of Artificial Creativity and Intelligence

Observing that the creation of certain types of artistic artifacts neces...

Human or Machine: Reflections on Turing-Inspired Testing for the Everyday

Turing's 1950 paper introduced the famed "imitation game", a test origin...

Toward a Human-Level Video Understanding Intelligence

We aim to develop an AI agent that can watch video clips and have a conv...

The Limitations of Standardized Science Tests as Benchmarks for Artificial Intelligence Research: Position Paper

In this position paper, I argue that standardized tests for elementary s...

Turing Test Revisited: A Framework for an Alternative

This paper aims to question the suitability of the Turing Test, for test...

Turing: Then, Now and Still Key

This paper looks at Turing's postulations about Artificial Intelligence ...