Sharp Asymptotics and Optimal Performance for Inference in Binary Models

02/17/2020
by   Hossein Taheri, et al.
0

We study convex empirical risk minimization for high-dimensional inference in binary models. Our first result sharply predicts the statistical performance of such estimators in the linear asymptotic regime under isotropic Gaussian features. Importantly, the predictions hold for a wide class of convex loss functions, which we exploit in order to prove a bound on the best achievable performance among them. Notably, we show that the proposed bound is tight for popular binary models (such as Signed, Logistic or Probit), by constructing appropriate loss functions that achieve it. More interestingly, for binary linear classification under the Logistic and Probit models, we prove that the performance of least-squares is no worse than 0.997 and 0.98 times the optimal one. Numerical simulations corroborate our theoretical findings and suggest they are accurate even for relatively small problem dimensions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/12/2019

Sharp Guarantees for Solving Random Equations with One-Bit Information

We study the performance of a wide class of convex optimization-based es...
research
06/16/2020

Fundamental Limits of Ridge-Regularized Empirical Risk Minimization in High Dimensions

Empirical Risk Minimization (ERM) algorithms are widely used in a variet...
research
04/13/2018

Robust 1-Bit Compressed Sensing via Hinge Loss Minimization

This work theoretically studies the problem of estimating a structured h...
research
01/18/2014

Excess Risk Bounds for Exponentially Concave Losses

The overarching goal of this paper is to derive excess risk bounds for l...
research
04/05/2023

Optimal Sketching Bounds for Sparse Linear Regression

We study oblivious sketching for k-sparse linear regression under variou...
research
10/16/2020

Binary Choice with Asymmetric Loss in a Data-Rich Environment: Theory and an Application to Racial Justice

The importance of asymmetries in prediction problems arising in economic...
research
10/09/2019

Learning Near-optimal Convex Combinations of Basis Models with Generalization Guarantees

The problem of learning an optimal convex combination of basis models ha...

Please sign up or login with your details

Forgot password? Click here to reset