DeepAI AI Chat
Log In Sign Up

Estimating decision tree learnability with polylogarithmic sample complexity

11/03/2020
by   Guy Blanc, et al.
0

We show that top-down decision tree learning heuristics are amenable to highly efficient learnability estimation: for monotone target functions, the error of the decision tree hypothesis constructed by these heuristics can be estimated with polylogarithmically many labeled examples, exponentially smaller than the number necessary to run these heuristics, and indeed, exponentially smaller than information-theoretic minimum required to learn a good decision tree. This adds to a small but growing list of fundamental learning algorithms that have been shown to be amenable to learnability estimation. En route to this result, we design and analyze sample-efficient minibatch versions of top-down decision tree learning heuristics and show that they achieve the same provable guarantees as the full-batch versions. We further give "active local" versions of these heuristics: given a test point x^⋆, we show how the label T(x^⋆) of the decision tree hypothesis T can be computed with polylogarithmically many labeled examples, exponentially smaller than the number necessary to learn T.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/01/2020

Provable guarantees for decision tree induction: the agnostic setting

We give strengthened provable guarantees on the performance of widely em...
10/16/2020

Universal guarantees for decision tree induction via a higher-order splitting criterion

We propose a simple extension of top-down decision tree learning heurist...
06/17/2022

Popular decision tree algorithms are provably noise tolerant

Using the framework of boosting, we prove that all impurity-based decisi...
07/02/2021

Decision tree heuristics can fail, even in the smoothed setting

Greedy decision tree learning heuristics are mainstays of machine learni...
05/08/2021

Learning stochastic decision trees

We give a quasipolynomial-time algorithm for learning stochastic decisio...
10/19/2017

E3Solver: decision tree unification by enumeration

We introduce E3Solver, a unification-based solver for programming-by-exa...
06/03/2011

An Analysis of Reduced Error Pruning

Top-down induction of decision trees has been observed to suffer from th...