The no-free-lunch theorems of supervised learning

02/09/2022
by   Tom F. Sterkenburg, et al.
0

The no-free-lunch theorems promote a skeptical conclusion that all possible machine learning algorithms equally lack justification. But how could this leave room for a learning theory, that shows that some algorithms are better than others? Drawing parallels to the philosophy of induction, we point out that the no-free-lunch results presuppose a conception of learning algorithms as purely data-driven. On this conception, every algorithm must have an inherent inductive bias, that wants justification. We argue that many standard learning algorithms should rather be understood as model-dependent: in each application they also require for input a model, representing a bias. Generic algorithms themselves, they can be given a model-relative justification.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/22/2021

The Implications of the No-Free-Lunch Theorems for Meta-induction

The important recent book by G. Schurz appreciates that the no-free-lunc...
research
12/10/2002

How to Shift Bias: Lessons from the Baldwin Effect

An inductive learning algorithm takes a set of data as input and generat...
research
04/11/2023

The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning

No free lunch theorems for supervised learning state that no learner can...
research
07/21/2020

What is important about the No Free Lunch theorems?

The No Free Lunch theorems prove that under a uniform distribution over ...
research
11/25/2020

On limitations of learning algorithms in competitive environments

We discuss conceptual limitations of generic learning algorithms acting ...
research
08/21/2021

Searching for a practical evidence of the No Free Lunch theorems

According to the No Free Lunch (NFL) theorems all black-box algorithms p...
research
08/26/2022

On the Implicit Bias in Deep-Learning Algorithms

Gradient-based deep-learning algorithms exhibit remarkable performance i...

Please sign up or login with your details

Forgot password? Click here to reset