Convex and Non-convex Approaches for Statistical Inference with Noisy Labels

by   Hyebin Song, et al.

We study the problem of estimation and testing in logistic regression with class-conditional noise in the observed labels, which has an important implication in the Positive-Unlabeled (PU) learning setting. With the key observation that the label noise problem belongs to a special sub-class of generalized linear models (GLM), we discuss convex and non-convex approaches that address this problem. A non-convex approach based on the maximum likelihood estimation produces an estimator with several optimal properties, but a convex approach has an obvious advantage in optimization. We demonstrate that in the low-dimensional setting, both estimators are consistent and asymptotically normal, where the asymptotic variance of the non-convex estimator is smaller than the convex counterpart. We also quantify the efficiency gap which provides insight into when the two methods are comparable. In the high-dimensional setting, we show that both estimation procedures achieve l2-consistency at the minimax optimal s log p/n rates under mild conditions. Finally, we propose an inference procedure using a de-biasing approach. To the best of our knowledge, this is the first work providing point estimation guarantees and hypothesis testing results for GLMs with non-canonical link functions, which is of independent interest. We validate our theoretical findings through simulations and a real-data example.


page 1

page 2

page 3

page 4


Sparse Tensor Graphical Model: Non-convex Optimization and Statistical Inference

We consider the estimation and inference of sparse graphical models that...

High-dimensional inference for inhomogeneous Gibbs point processes

Gibbs point processes (GPPs) constitute a large and flexible class of sp...

Statistical Inference in High-Dimensional Generalized Linear Models with Asymmetric Link Functions

We have developed a statistical inference method applicable to a broad r...

Misspecification Analysis of High-Dimensional Random Effects Models for Estimation of Signal-to-Noise Ratios

Estimation of signal-to-noise ratios and noise variances in high-dimensi...

An equivalence between high dimensional Bayes optimal inference and M-estimation

When recovering an unknown signal from noisy measurements, the computati...

Confidence Sets under Generalized Self-Concordance

This paper revisits a fundamental problem in statistical inference from ...

Statistical Hypothesis Testing for Class-Conditional Label Noise

In this work we aim to provide machine learning practitioners with tools...

Please sign up or login with your details

Forgot password? Click here to reset