Statistical Hypothesis Testing for Class-Conditional Label Noise

03/03/2021
by   Rafael Poyiadzi, et al.
7

In this work we aim to provide machine learning practitioners with tools to answer the question: is there class-conditional flipping noise in my labels? In particular, we present hypothesis tests to reliably check whether a given dataset of instance-label pairs has been corrupted with class-conditional label noise. While previous works explore the direct estimation of the noise rates, this is known to be hard in practice and does not offer a real understanding of how trustworthy the estimates are. These methods typically require anchor points - examples whose true posterior is either 0 or 1. Differently, in this paper we assume we have access to a set of anchor points whose true posterior is approximately 1/2. The proposed hypothesis tests are built upon the asymptotic properties of Maximum Likelihood Estimators for Logistic Regression models and accurately distinguish the presence of class-conditional noise from uniform noise. We establish the main properties of the tests, including a theoretical and empirical analysis of the dependence of the power on the test on the training sample size, the number of anchor points, the difference of the noise rates and the use of realistic relaxed anchors.

READ FULL TEXT
research
12/16/2022

Instance-specific Label Distribution Regularization for Learning with Label Noise

Modeling noise transition matrix is a kind of promising method for learn...
research
12/10/2020

Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise

Supervised learning under label noise has seen numerous advances recentl...
research
09/14/2023

Anchor Points: Benchmarking Models with Much Fewer Examples

Modern language models often exhibit powerful but brittle behavior, lead...
research
05/16/2021

CCMN: A General Framework for Learning with Class-Conditional Multi-Label Noise

Class-conditional noise commonly exists in machine learning tasks, where...
research
10/06/2019

Convex and Non-convex Approaches for Statistical Inference with Noisy Labels

We study the problem of estimation and testing in logistic regression wi...
research
03/05/2013

Classification with Asymmetric Label Noise: Consistency and Maximal Denoising

In many real-world classification problems, the labels of training examp...
research
08/04/2023

FPR Estimation for Fraud Detection in the Presence of Class-Conditional Label Noise

We consider the problem of estimating the false-/ true-positive-rate (FP...

Please sign up or login with your details

Forgot password? Click here to reset