Confidence Decision Trees via Online and Active Learning for Streaming (BIG) Data

04/12/2016
by   Rocco De Rosa, et al.
0

Decision tree classifiers are a widely used tool in data stream mining. The use of confidence intervals to estimate the gain associated with each split leads to very effective methods, like the popular Hoeffding tree algorithm. From a statistical viewpoint, the analysis of decision tree classifiers in a streaming setting requires knowing when enough new information has been collected to justify splitting a leaf. Although some of the issues in the statistical analysis of Hoeffding trees have been already clarified, a general and rigorous study of confidence intervals for splitting criteria is missing. We fill this gap by deriving accurate confidence intervals to estimate the splitting gain in decision tree learning with respect to three criteria: entropy, Gini index, and a third index proposed by Kearns and Mansour. Our confidence intervals depend in a more detailed way on the tree parameters. We also extend our confidence analysis to a selective sampling setting, in which the decision tree learner adaptively decides which labels to query in the stream. We furnish theoretical guarantee bounding the probability that the classification is non-optimal learning the decision tree via our selective sampling strategy. Experiments on real and synthetic data in a streaming setting show that our trees are indeed more accurate than trees with the same number of leaves generated by other techniques and our active learning module permits to save labeling cost. In addition, comparing our labeling strategy with recent methods, we show that our approach is more robust and consistent respect all the other techniques applied to incremental decision trees.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/12/2015

Efficient non-greedy optimization of decision trees

Decision trees and randomized forests are widely used in computer vision...
research
01/02/2020

Explainable outlier detection through decision tree conditioning

This work describes an outlier detection procedure (named "OutlierTree")...
research
10/20/2020

An Eager Splitting Strategy for Online Decision Trees

We study the effectiveness of replacing the split strategy for the state...
research
11/25/2015

Unifying Decision Trees Split Criteria Using Tsallis Entropy

The construction of efficient and effective decision trees remains a key...
research
11/17/2012

Cost-sensitive C4.5 with post-pruning and competition

Decision tree is an effective classification approach in data mining and...
research
10/16/2020

Universal guarantees for decision tree induction via a higher-order splitting criterion

We propose a simple extension of top-down decision tree learning heurist...
research
08/03/2023

Multitask Learning with No Regret: from Improved Confidence Bounds to Active Learning

Multitask learning is a powerful framework that enables one to simultane...

Please sign up or login with your details

Forgot password? Click here to reset