
Improved Rates for Differentially Private Stochastic Convex Optimization with HeavyTailed Data
We study stochastic convex optimization with heavytailed data under the...
read it

Faster Rates of Differentially Private Stochastic Convex Optimization
In this paper, we revisit the problem of Differentially Private Stochast...
read it

Private Nonsmooth Empirical Risk Minimization and Stochastic Convex Optimization in Subquadratic Steps
We study the differentially private Empirical Risk Minimization (ERM) an...
read it

Private Stochastic Convex Optimization with Optimal Rates
We study differentially private (DP) algorithms for stochastic convex op...
read it

Private Stochastic Convex Optimization: Optimal Rates in Linear Time
We study differentially private (DP) algorithms for stochastic convex op...
read it

Differentially Private SGD with NonSmooth Loss
In this paper, we are concerned with differentially private SGD algorith...
read it

Shuffle Private Stochastic Convex Optimization
In shuffle privacy, each user sends a collection of randomized messages ...
read it
Private Stochastic Convex Optimization: Optimal Rates in ℓ_1 Geometry
Stochastic convex optimization over an ℓ_1bounded domain is ubiquitous in machine learning applications such as LASSO but remains poorly understood when learning with differential privacy. We show that, up to logarithmic factors the optimal excess population loss of any (ε,δ)differentially private optimizer is √(log(d)/n) + √(d)/ε n. The upper bound is based on a new algorithm that combines the iterative localization approach of <cit.> with a new analysis of private regularized mirror descent. It applies to ℓ_p bounded domains for p∈ [1,2] and queries at most n^3/2 gradients improving over the best previously known algorithm for the ℓ_2 case which needs n^2 gradients. Further, we show that when the loss functions satisfy additional smoothness assumptions, the excess loss is upper bounded (up to logarithmic factors) by √(log(d)/n) + (log(d)/ε n)^2/3. This bound is achieved by a new variancereduced version of the FrankWolfe algorithm that requires just a single pass over the data. We also show that the lower bound in this case is the minimum of the two rates mentioned above.
READ FULL TEXT
Comments
There are no comments yet.