DeepAI AI Chat
Log In Sign Up

The Implications of the No-Free-Lunch Theorems for Meta-induction

by   David H. Wolpert, et al.

The important recent book by G. Schurz appreciates that the no-free-lunch theorems (NFL) have major implications for the problem of (meta) induction. Here I review the NFL theorems, emphasizing that they do not only concern the case where there is a uniform prior – they prove that there are “as many priors” (loosely speaking) for which any induction algorithm A out-generalizes some induction algorithm B as vice-versa. Importantly though, in addition to the NFL theorems, there are many free lunch theorems. In particular, the NFL theorems can only be used to compare the marginal expected performance of an induction algorithm A with the marginal expected performance of an induction algorithm B. There is a rich set of free lunches which instead concern the statistical correlations among the generalization errors of induction algorithms. As I describe, the meta-induction algorithms that Schurz advocate as a “solution to Hume's problem” are just an example of such a free lunch based on correlations among the generalization errors of induction algorithms. I end by pointing out that the prior that Schurz advocates, which is uniform over bit frequencies rather than bit patterns, is contradicted by thousands of experiments in statistical physics and by the great success of the maximum entropy procedure in inductive inference.


page 1

page 2

page 3

page 4


What is important about the No Free Lunch theorems?

The No Free Lunch theorems prove that under a uniform distribution over ...

The no-free-lunch theorems of supervised learning

The no-free-lunch theorems promote a skeptical conclusion that all possi...

Quantifier-free induction for lists

We investigate quantifier-free induction for Lisp-like lists constructed...

Interpolating Strong Induction

The principle of strong induction, also known as k-induction is one of t...

A Novice-Friendly Induction Tactic for Lean

In theorem provers based on dependent type theory such as Coq and Lean, ...

Induction, Popper, and machine learning

Francis Bacon popularized the idea that science is based on a process of...

Induction, of and by Probability

This paper examines some methods and ideas underlying the author's succe...