Differentially Private Empirical Risk Minimization Revisited: Faster and More General

02/14/2018
by   Di Wang, et al.
0

In this paper we study the differentially private Empirical Risk Minimization (ERM) problem in different settings. For smooth (strongly) convex loss function with or without (non)-smooth regularization, we give algorithms that achieve either optimal or near optimal utility bounds with less gradient complexity compared with previous work. For ERM with smooth convex loss function in high-dimensional (p≫ n) setting, we give an algorithm which achieves the upper bound with less gradient complexity than previous ones. At last, we generalize the expected excess empirical risk from convex loss functions to non-convex ones satisfying the Polyak-Lojasiewicz condition and give a tighter upper bound on the utility than the one in ijcai2017-548.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/29/2021

Private Non-smooth Empirical Risk Minimization and Stochastic Convex Optimization in Subquadratic Steps

We study the differentially private Empirical Risk Minimization (ERM) an...
research
05/27/2014

Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds

In this paper, we initiate a systematic investigation of differentially ...
research
02/24/2023

Differentially Private Algorithms for the Stochastic Saddle Point Problem with Optimal Rates for the Strong Gap

We show that convex-concave Lipschitz stochastic saddle point problems (...
research
06/02/2021

Statistical optimality conditions for compressive ensembles

We present a framework for the theoretical analysis of ensembles of low-...
research
05/19/2021

Localization, Convexity, and Star Aggregation

Offset Rademacher complexities have been shown to imply sharp, data-depe...
research
11/30/2020

Persistent Reductions in Regularized Loss Minimization for Variable Selection

In the context of regularized loss minimization with polyhedral gauges, ...
research
11/20/2014

Private Empirical Risk Minimization Beyond the Worst Case: The Effect of the Constraint Set Geometry

Empirical Risk Minimization (ERM) is a standard technique in machine lea...

Please sign up or login with your details

Forgot password? Click here to reset