DeepAI AI Chat
Log In Sign Up

All Mistakes Are Not Equal: Comprehensive Hierarchy Aware Multi-label Predictions (CHAMP)

by   Ashwin Vaswani, et al.

This paper considers the problem of Hierarchical Multi-Label Classification (HMC), where (i) several labels can be present for each example, and (ii) labels are related via a domain-specific hierarchy tree. Guided by the intuition that all mistakes are not equal, we present Comprehensive Hierarchy Aware Multi-label Predictions (CHAMP), a framework that penalizes a misprediction depending on its severity as per the hierarchy tree. While there have been works that apply such an idea to single-label classification, to the best of our knowledge, there are limited such works for multilabel classification focusing on the severity of mistakes. The key reason is that there is no clear way of quantifying the severity of a misprediction a priori in the multilabel setting. In this work, we propose a simple but effective metric to quantify the severity of a mistake in HMC, naturally leading to CHAMP. Extensive experiments on six public HMC datasets across modalities (image, audio, and text) demonstrate that incorporating hierarchical information leads to substantial gains as CHAMP improves both AUPRC (2.6 median percentage improvement) and hierarchical metrics (2.85 percentage improvement), over stand-alone hierarchical or multilabel classification methods. Compared to standard multilabel baselines, CHAMP provides improved AUPRC in both robustness (8.87 and less data regimes. Further, our method provides a framework to enhance existing multilabel classification algorithms with better mistakes (18.1 percentage increment).


page 15

page 16


Coherent Hierarchical Multi-Label Classification Networks

Hierarchical multi-label classification (HMC) is a challenging classific...

Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text Classification

Hierarchical text classification is a challenging subtask of multi-label...

No Cost Likelihood Manipulation at Test Time for Making Better Mistakes in Deep Networks

There has been increasing interest in building deep hierarchy-aware clas...

HiTIN: Hierarchy-aware Tree Isomorphism Network for Hierarchical Text Classification

Hierarchical text classification (HTC) is a challenging subtask of multi...

Joint Learning of Hyperbolic Label Embeddings for Hierarchical Multi-label Classification

We consider the problem of multi-label classification where the labels l...

A Hierarchical Assessment of Adversarial Severity

Adversarial Robustness is a growing field that evidences the brittleness...

Severity Level of Permissions in Role-Based Access Control

The analysis of hidden channels of information leakage with respect to r...