Robust high dimensional learning for Lipschitz and convex losses

05/10/2019
by   Geoffrey Chinot, et al.
0

We establish risk bounds for Regularized Empirical Risk Minimizers (RERM) when the loss is Lipschitz and convex and the regularization function is a norm. We obtain these results in the i.i.d. setup under subgaussian assumptions on the design. In a second part, a more general framework where the design might have heavier tails and data may be corrupted by outliers both in the design and the response variables is considered. In this situation, RERM performs poorly in general. We analyse an alternative procedure based on median-of-means principles and called "minmax MOM". We show optimal subgaussian deviation rates for these estimators in the relaxed setting. The main results are meta-theorems allowing a wide-range of applications to various problems in learning theory. To show a non-exhaustive sample of these potential applications, it is applied to classification problems with logistic loss functions regularized by LASSO and SLOPE, to regression problems with Huber loss regularized by Group LASSO, Total Variation and Fused LASSO and to matrix completion problems with quantile loss regularized by the nuclear norm. A short simulation study concludes the paper, illustrating in particular robustness properties of regularized minmax MOM procedures.

READ FULL TEXT
research
10/24/2019

ERM and RERM are optimal estimators for regression problems when malicious outliers corrupt the labels

We study Empirical Risk Minimizers (ERM) and Regularized Empirical Risk ...
research
02/06/2019

Robust learning and complexity dependent bounds for regularized problems

We obtain risk bounds for Regularized Empirical Risk Minimizers (RERM) a...
research
06/18/2020

Median Matrix Completion: from Embarrassment to Optimality

In this paper, we consider matrix completion with absolute deviation los...
research
07/14/2018

Sparse Relaxed Regularized Regression: SR3

Regularized regression problems are ubiquitous in statistical modeling, ...
research
10/02/2018

Statistical learning with Lipschitz and convex loss functions

We obtain risk bounds for Empirical Risk Minimizers (ERM) and minmax Med...
research
01/15/2017

Regularization, sparse recovery, and median-of-means tournaments

A regularized risk minimization procedure for regression function estima...
research
12/06/2018

Median of means principle as a divide-and-conquer procedure for robustness, sub-sampling and hyper-parameters tuning

Many learning methods have poor risk estimates with large probability un...

Please sign up or login with your details

Forgot password? Click here to reset