Renyi Differentially Private ADMM for Non-Smooth Regularized Optimization

09/18/2019
by   Chen Chen, et al.
0

In this paper we consider the problem of minimizing composite objective functions consisting of a convex differentiable loss function plus a non-smooth regularization term, such as L_1 norm or nuclear norm, under Rényi differential privacy (RDP). To solve the problem, we propose two stochastic alternating direction method of multipliers (ADMM) algorithms: ssADMM based on gradient perturbation and mpADMM based on output perturbation. Both algorithms decompose the original problem into sub-problems that have closed-form solutions. The first algorithm, ssADMM, applies the recent privacy amplification result for RDP to reduce the amount of noise to add. The second algorithm, mpADMM, numerically computes the sensitivity of ADMM variable updates and releases the updated parameter vector at the end of each epoch. We compare the performance of our algorithms with several baseline algorithms on both real and simulated datasets. Experimental results show that, in high privacy regimes (small ϵ), ssADMM and mpADMM outperform other baseline algorithms in terms of classification and feature selection performance, respectively.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 11

09/18/2019

Renyi Differentially Private ADMM Based L1 Regularized Classification

In this paper we present two new algorithms, to solve the L1 regularized...
10/31/2020

Differentially Private ADMM Algorithms for Machine Learning

In this paper, we study efficient differentially private alternating dir...
11/03/2012

Stochastic ADMM for Nonsmooth Optimization

We present a stochastic setting for optimization problems with nonsmooth...
08/30/2018

DP-ADMM: ADMM-based Distributed Learning with Differential Privacy

Distributed machine learning is making great changes in a wide variety o...
06/02/2012

Sparse Trace Norm Regularization

We study the problem of estimating multiple predictive functions from a ...
08/02/2019

Differential Privacy for Sparse Classification Learning

In this paper, we present a differential privacy version of convex and n...
03/10/2017

Tuning Over-Relaxed ADMM

The framework of Integral Quadratic Constraints (IQC) reduces the comput...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.