Faster Rates of Differentially Private Stochastic Convex Optimization

07/31/2021
by   Jinyan Su, et al.
0

In this paper, we revisit the problem of Differentially Private Stochastic Convex Optimization (DP-SCO) and provide excess population risks for some special classes of functions that are faster than the previous results of general convex and strongly convex functions. In the first part of the paper, we study the case where the population risk function satisfies the Tysbakov Noise Condition (TNC) with some parameter θ>1. Specifically, we first show that under some mild assumptions on the loss functions, there is an algorithm whose output could achieve an upper bound of Õ((1/√(n)+√(dlog1/δ)/nϵ)^θ/θ-1) for (ϵ, δ)-DP when θ≥ 2, here n is the sample size and d is the dimension of the space. Then we address the inefficiency issue, improve the upper bounds by Poly(log n) factors and extend to the case where θ≥θ̅>1 for some known θ̅. Next we show that the excess population risk of population functions satisfying TNC with parameter θ>1 is always lower bounded by Ω((d/nϵ)^θ/θ-1) and Ω((√(dlog1/δ)/nϵ)^θ/θ-1) for ϵ-DP and (ϵ, δ)-DP, respectively. In the second part, we focus on a special case where the population risk function is strongly convex. Unlike the previous studies, here we assume the loss function is non-negative and the optimal value of population risk is sufficiently small. With these additional assumptions, we propose a new method whose output could achieve an upper bound of O(dlog1/δ/n^2ϵ^2+1/n^τ) for any τ≥ 1 in (ϵ,δ)-DP model if the sample size n is sufficiently large.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/21/2020

On Differentially Private Stochastic Convex Optimization with Heavy-tailed Data

In this paper, we consider the problem of designing Differentially Priva...
research
05/28/2021

Curse of Dimensionality in Unconstrained Private Convex ERM

We consider the lower bounds of differentially private empirical risk mi...
research
03/31/2023

Differentially Private Stochastic Convex Optimization in (Non)-Euclidean Space Revisited

In this paper, we revisit the problem of Differentially Private Stochast...
research
03/02/2021

Private Stochastic Convex Optimization: Optimal Rates in ℓ_1 Geometry

Stochastic convex optimization over an ℓ_1-bounded domain is ubiquitous ...
research
06/02/2022

Faster Rates of Convergence to Stationary Points in Differentially Private Optimization

We study the problem of approximating stationary points of Lipschitz and...
research
06/02/2021

Statistical optimality conditions for compressive ensembles

We present a framework for the theoretical analysis of ensembles of low-...

Please sign up or login with your details

Forgot password? Click here to reset