Bad Universal Priors and Notions of Optimality

10/16/2015
by   Jan Leike, et al.
0

A big open question of algorithmic information theory is the choice of the universal Turing machine (UTM). For Kolmogorov complexity and Solomonoff induction we have invariance theorems: the choice of the UTM changes bounds only by a constant. For the universally intelligent agent AIXI (Hutter, 2005) no invariance theorem is known. Our results are entirely negative: we discuss cases in which unlucky or adversarial choices of the UTM cause AIXI to misbehave drastically. We show that Legg-Hutter intelligence and thus balanced Pareto optimality is entirely subjective, and that every policy is Pareto optimal in the class of all computable environments. This undermines all existing optimality properties for AIXI. While it may still serve as a gold standard for AI, our results imply that AIXI is a relative theory, dependent on the choice of the UTM.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

11/28/2016

Nonparametric General Reinforcement Learning

Reinforcement learning (RL) problems are often phrased in terms of Marko...
07/04/2009

Open Problems in Universal Induction & Intelligence

Specialized intelligent systems can be found everywhere: finger print, h...
02/14/2020

An Optimal Procedure to Check Pareto-Optimality in House Markets with Single-Peaked Preferences

Recently, the problem of allocating one resource per agent with initial ...
09/19/2012

Cognitive Bias for Universal Algorithmic Intelligence

Existing theoretical universal algorithmic intelligence models are not p...
10/12/2018

Non-computability of human intelligence

We revisit the question (most famously) initiated by Turing:Can human in...
08/02/2021

Escaping Arrow's Theorem: The Advantage-Standard Model

There is an extensive literature in social choice theory studying the co...
06/01/2013

Universal Induction with Varying Sets of Combinators

Universal induction is a crucial issue in AGI. Its practical applicabili...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.