Gradient-based Label Binning in Multi-label Classification

06/22/2021
by   Michael Rapp, et al.
0

In multi-label classification, where a single example may be associated with several class labels at the same time, the ability to model dependencies between labels is considered crucial to effectively optimize non-decomposable evaluation measures, such as the Subset 0/1 loss. The gradient boosting framework provides a well-studied foundation for learning models that are specifically tailored to such a loss function and recent research attests the ability to achieve high predictive accuracy in the multi-label setting. The utilization of second-order derivatives, as used by many recent boosting approaches, helps to guide the minimization of non-decomposable losses, due to the information about pairs of labels it incorporates into the optimization process. On the downside, this comes with high computational costs, even if the number of labels is small. In this work, we address the computational bottleneck of such approach – the need to solve a system of linear equations – by integrating a novel approximation technique into the boosting procedure. Based on the derivatives computed during training, we dynamically group the labels into a predefined number of bins to impose an upper bound on the dimensionality of the linear system. Our experiments, using an existing rule-based algorithm, suggest that this may boost the speed of training, without any significant loss in predictive performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/23/2020

Learning Gradient Boosted Multi-label Classification Rules

In multi-label classification, where the evaluation of predictions is le...
research
08/19/2019

Efficient Discovery of Expressive Multi-label Rules using Relaxed Pruning

Being able to model correlations between labels is considered crucial in...
research
10/04/2022

Concise and interpretable multi-label rule sets

Multi-label classification is becoming increasingly ubiquitous, but not ...
research
10/23/2017

Online Boosting Algorithms for Multi-label Ranking

We consider the multi-label ranking approach to multi-label learning. Bo...
research
06/15/2020

Extreme Gradient Boosted Multi-label Trees for Dynamic Classifier Chains

Classifier chains is a key technique in multi-label classification, sinc...
research
09/27/2021

Speeding-up One-vs-All Training for Extreme Classification via Smart Initialization

In this paper we show that a simple, data dependent way of setting the i...
research
09/11/2020

Deep Hiearchical Multi-Label Classification Applied to Chest X-Ray Abnormality Taxonomies

CXRs are a crucial and extraordinarily common diagnostic tool, leading t...

Please sign up or login with your details

Forgot password? Click here to reset