Rethinking and Reweighting the Univariate Losses for Multi-Label Ranking: Consistency and Generalization

05/10/2021
by   Guoqiang Wu, et al.
0

(Partial) ranking loss is a commonly used evaluation measure for multi-label classification, which is usually optimized with convex surrogates for computational efficiency. Prior theoretical work on multi-label ranking mainly focuses on (Fisher) consistency analyses. However, there is a gap between existing theory and practice – some pairwise losses can lead to promising performance but lack consistency, while some univariate losses are consistent but usually have no clear superiority in practice. In this paper, we attempt to fill this gap through a systematic study from two complementary perspectives of consistency and generalization error bounds of learning algorithms. Our results show that learning algorithms with the consistent univariate loss have an error bound of O(c) (c is the number of labels), while algorithms with the inconsistent pairwise loss depend on O(√(c)) as shown in prior work. This explains that the latter can achieve better performance than the former in practice. Moreover, we present an inconsistent reweighted univariate loss-based learning algorithm that enjoys an error bound of O(√(c)) for promising performance as well as the computational efficiency of univariate losses. Finally, experimental results validate our theoretical analyses.

READ FULL TEXT
research
11/16/2020

Multi-label classification: do Hamming loss and subset accuracy really conflict with each other?

Various evaluation measures have been developed for multi-label classifi...
research
05/09/2023

Towards Understanding Generalization of Macro-AUC in Multi-label Learning

Macro-AUC is the arithmetic mean of the class-wise AUCs in multi-label l...
research
10/16/2018

Sharp Analysis of Learning with Discrete Losses

The problem of devising learning strategies for discrete losses (e.g., m...
research
06/27/2012

Consistent Multilabel Ranking through Univariate Losses

We consider the problem of rank loss minimization in the setting of mult...
research
04/11/2017

Improving Pairwise Ranking for Multi-label Image Classification

Learning to rank has recently emerged as an attractive technique to trai...
research
12/01/2021

On the Practical Consistency of Meta-Reinforcement Learning Algorithms

Consistency is the theoretical property of a meta learning algorithm tha...
research
10/26/2018

Quantifying Learning Guarantees for Convex but Inconsistent Surrogates

We study consistency properties of machine learning methods based on min...

Please sign up or login with your details

Forgot password? Click here to reset