Categorizing Variants of Goodhart's Law

03/13/2018
by   David Manheim, et al.
0

There are several distinct failure modes for overoptimization of systems on the basis of metrics. This occurs when a metric which can be used to improve a system is used to an extent that further optimization is ineffective or harmful, and is sometimes termed Goodhart's Law. This class of failure is often poorly understood, partly because terminology for discussing them is ambiguous, and partly because discussion using this ambiguous terminology ignores distinctions between different failure modes of this general type. This paper expands on an earlier discussion by Garrabrant, which notes there are "(at least) four different mechanisms" that relate to Goodhart's Law. This paper is intended to explore these mechanisms further, and specify more clearly how they occur. This discussion should be helpful in better understanding these types of failures in economic regulation, in public policy, in machine learning, and in Artificial Intelligence alignment. The importance of Goodhart effects depends on the amount of power directed towards optimizing the proxy, and so the increased optimization power offered by artificial intelligence makes it especially critical for that field.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/16/2018

Multiparty Dynamics and Failure Modes for Machine Learning and Artificial Intelligence

Overoptimization failures in machine learning and artificial intelligenc...
research
10/16/2018

Overoptimization Failures and Specification Gaming in Multi-agent Systems

Overoptimization failures in machine learning and AI can involve specifi...
research
11/15/2022

Power-law Scaling to Assist with Key Challenges in Artificial Intelligence

Power-law scaling, a central concept in critical phenomena, is found to ...
research
02/04/2019

Evaluation of Multidisciplinary Effects of Artificial Intelligence with Optimization Perspective

Artificial Intelligence has an important place in the scientific communi...
research
03/11/2014

Turing: Then, Now and Still Key

This paper looks at Turing's postulations about Artificial Intelligence ...
research
08/12/2019

A modified Coulomb's law for the tangential debonding of osseointegrated implants

Cementless implants are widely used in orthopedic and oral surgery. Howe...

Please sign up or login with your details

Forgot password? Click here to reset