Evaluating Multi-label Classifiers with Noisy Labels

02/16/2021
by   Wenting Zhao, et al.
17

Multi-label classification (MLC) is a generalization of standard classification where multiple labels may be assigned to a given sample. In the real world, it is more common to deal with noisy datasets than clean datasets, given how modern datasets are labeled by a large group of annotators on crowdsourcing platforms, but little attention has been given to evaluating multi-label classifiers with noisy labels. Exploiting label correlations now becomes a standard component of a multi-label classifier to achieve competitive performance. However, this component makes the classifier more prone to poor generalization - it overfits labels as well as label dependencies. We identify three common real-world label noise scenarios and show how previous approaches per-form poorly with noisy labels. To address this issue, we present a Context-Based Multi-LabelClassifier (CbMLC) that effectively handles noisy labels when learning label dependencies, without requiring additional supervision. We compare CbMLC against other domain-specific state-of-the-art models on a variety of datasets, under both the clean and the noisy settings. We show CbMLC yields substantial improvements over the previous methods in most cases.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/03/2022

Semantic-guided Image Virtual Attribute Learning for Noisy Multi-label Chest X-ray Classification

Deep learning methods have shown outstanding classification accuracy in ...
research
06/19/2017

Multi-Label Annotation Aggregation in Crowdsourcing

As a means of human-based computation, crowdsourcing has been widely use...
research
12/05/2012

Evaluating Classifiers Without Expert Labels

This paper considers the challenge of evaluating a set of classifiers, a...
research
07/12/2020

Disentangled Variational Autoencoder based Multi-Label Classification with Covariance-Aware Multivariate Probit Model

Multi-label classification is the challenging task of predicting the pre...
research
11/14/2022

Multi-Label Training for Text-Independent Speaker Identification

In this paper, we propose a novel strategy for text-independent speaker ...
research
02/17/2022

PENCIL: Deep Learning with Noisy Labels

Deep learning has achieved excellent performance in various computer vis...
research
04/19/2016

Streaming Label Learning for Modeling Labels on the Fly

It is challenging to handle a large volume of labels in multi-label lear...

Please sign up or login with your details

Forgot password? Click here to reset