-
Improving Differentially Private Models with Active Learning
Broad adoption of machine learning techniques has increased privacy conc...
10/02/2019 ∙ by Zhengli Zhao, et al. ∙
26
∙
share
read it
-
Differentially Private Distributed Learning for Language Modeling Tasks
One of the big challenges in machine learning applications is that train...
12/20/2017 ∙ by Vadim Popov, et al. ∙
0
∙
share
read it
-
Privacy Analysis in Language Models via Training Data Leakage Report
Recent advances in neural network based language models lead to successf...
01/14/2021 ∙ by Huseyin A. Inan, et al. ∙
0
∙
share
read it
-
Learning to Customize Language Model for Generation-based dialog systems
Personalized conversation systems have received increasing attention rec...
10/31/2019 ∙ by Yiping Song, et al. ∙
0
∙
share
read it
-
No-Regret Algorithms for Private Gaussian Process Bandit Optimization
The widespread proliferation of data-driven decision-making has ushered ...
02/24/2021 ∙ by Abhimanyu Dubey, et al. ∙
0
∙
share
read it
-
Analyzing Privacy Loss in Updates of Natural Language Models
To continuously improve quality and reflect changes in data, machine lea...
12/17/2019 ∙ by Shruti Tople, et al. ∙
0
∙
share
read it
-
Privacy-Preserving Graph Convolutional Networks for Text Classification
Graph convolutional networks (GCNs) are a powerful architecture for repr...
02/10/2021 ∙ by Timour Igamberdiev, et al. ∙
0
∙
share
read it
Differentially Private Language Models Benefit from Public Pre-training
09/13/2020 ∙ by Gavin Kerrigan, et al. ∙
0
∙
share
Language modeling is a keystone task in natural language processing. When training a language model on sensitive information, differential privacy (DP) allows us to quantify the degree to which our private data is protected. However, training algorithms which enforce differential privacy often lead to degradation in model quality. We study the feasibility of learning a language model which is simultaneously high-quality and privacy preserving by tuning a public base model on a private corpus. We find that DP fine-tuning boosts the performance of language models in the private domain, making the training of such models possible.
READ FULL TEXT
Comments
There are no comments yet.