On the computational complexity of the probabilistic label tree algorithms

06/01/2019
by   Robert Busa-Fekete, et al.
0

Label tree-based algorithms are widely used to tackle multi-class and multi-label problems with a large number of labels. We focus on a particular subclass of these algorithms that use probabilistic classifiers in the tree nodes. Examples of such algorithms are hierarchical softmax (HSM), designed for multi-class classification, and probabilistic label trees (PLTs) that generalize HSM to multi-label problems. If the tree structure is given, learning of PLT can be solved with provable regret guaranties [Wydmuch et.al. 2018]. However, to find a tree structure that results in a PLT with a low training and prediction computational costs as well as low statistical error seems to be a very challenging problem, not well-understood yet. In this paper, we address the problem of finding a tree structure that has low computational cost. First, we show that finding a tree with optimal training cost is NP-complete, nevertheless there are some tractable special cases with either perfect approximation or exact solution that can be obtained in linear time in terms of the number of labels m. For the general case, we obtain O(log m) approximation in linear time too. Moreover, we prove an upper bound on the expected prediction cost expressed in terms of the expected training cost. We also show that under additional assumptions the prediction cost of a PLT is O(log m).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/23/2020

Probabilistic Label Trees for Extreme Multi-label Classification

Extreme multi-label classification (XMLC) is a learning task of tagging ...
research
10/07/2013

Least Squares Revisited: Scalable Approaches for Multi-class Prediction

This work provides simple algorithms for multi-class (and multi-label) p...
research
08/09/2014

Conditional Probability Tree Estimation Analysis and Algorithms

We consider the problem of estimating the conditional probability of a l...
research
07/08/2020

Online probabilistic label trees

We introduce online probabilistic label trees (OPLTs), an algorithm that...
research
10/14/2016

Simultaneous Learning of Trees and Representations for Extreme Classification and Density Estimation

We consider multi-class classification where the predictor has a hierarc...
research
02/03/2014

Transductive Learning with Multi-class Volume Approximation

Given a hypothesis space, the large volume principle by Vladimir Vapnik ...
research
07/17/2018

Contextual Memory Trees

We design and study a Contextual Memory Tree (CMT), a learning memory co...

Please sign up or login with your details

Forgot password? Click here to reset