DeepAI AI Chat
Log In Sign Up

Adaptive Private-K-Selection with Adaptive K and Application to Multi-label PATE

03/30/2022
by   Yuqing Zhu, et al.
0

We provide an end-to-end Renyi DP based-framework for differentially private top-k selection. Unlike previous approaches, which require a data-independent choice on k, we propose to privately release a data-dependent choice of k such that the gap between k-th and the (k+1)st "quality" is large. This is achieved by a novel application of the Report-Noisy-Max. Not only does this eliminate one hyperparameter, the adaptive choice of k also certifies the stability of the top-k indices in the unordered set so we can release them using a variant of propose-test-release (PTR) without adding noise. We show that our construction improves the privacy-utility trade-offs compared to the previous top-k selection algorithms theoretically and empirically. Additionally, we apply our algorithm to "Private Aggregation of Teacher Ensembles (PATE)" in multi-label classification tasks with a large number of labels and show that it leads to significant performance gains.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/31/2022

Generalized PTR: User-Friendly Recipes for Data-Adaptive Algorithms with Differential Privacy

The ”Propose-Test-Release” (PTR) framework is a classic recipe for desig...
12/01/2022

Differentially Private Adaptive Optimization with Delayed Preconditioners

Privacy noise may negate the benefits of using adaptive optimizers in di...
01/16/2022

Visualizing Privacy-Utility Trade-Offs in Differentially Private Data Releases

Organizations often collect private data and release aggregate statistic...
11/09/2021

The Role of Adaptive Optimizers for Honest Private Hyperparameter Selection

Hyperparameter optimization is a ubiquitous challenge in machine learnin...
11/22/2022

Generalized Private Selection and Testing with High Confidence

Composition theorems are general and powerful tools that facilitate priv...
04/29/2019

Free Gap Information from the Differentially Private Sparse Vector and Noisy Max Mechanisms

Noisy Max and Sparse Vector are selection algorithms for differential pr...