DeepAI AI Chat
Log In Sign Up

Revisiting Model-Agnostic Private Learning: Faster Rates and Active Learning

by   Chong Liu, et al.
The Regents of the University of California
University of California, San Diego

The Private Aggregation of Teacher Ensembles (PATE) framework is one of the most promising recent approaches in differentially private learning. Existing theoretical analysis shows that PATE consistently learns any VC-classes in the realizable setting, but falls short in explaining its success in more general cases where the error rate of the optimal classifier is bounded away from zero. We fill in this gap by introducing the Tsybakov Noise Condition (TNC) and establish stronger and more interpretable learning bounds. These bounds provide new insights into when PATE works and improve over existing results even in the narrower realizable setting. We also investigate the compelling idea of using active learning for saving privacy budget. The novel components in the proofs include a more refined analysis of the majority voting classifier – which could be of independent interest – and an observation that the synthetic "student" learning problem is nearly realizable by construction under the Tsybakov noise condition.


page 1

page 2

page 3

page 4


Statistical Active Learning Algorithms for Noise Tolerance and Differential Privacy

We describe a framework for designing efficient active learning algorith...

Refined Error Bounds for Several Learning Algorithms

This article studies the achievable guarantees on the error rates of cer...

Robust and Private Learning of Halfspaces

In this work, we study the trade-off between differential privacy and ad...

Differentially- and non-differentially-private random decision trees

We consider supervised learning with random decision trees, where the tr...

Model Conversion via Differentially Private Data-Free Distillation

While massive valuable deep models trained on large-scale data have been...

When Does Differentially Private Learning Not Suffer in High Dimensions?

Large pretrained models can be privately fine-tuned to achieve performan...

Near-Optimal Active Learning of Halfspaces via Query Synthesis in the Noisy Setting

In this paper, we consider the problem of actively learning a linear cla...