Privately Learning Subspaces

05/28/2021
by   Vikrant Singhal, et al.
0

Private data analysis suffers a costly curse of dimensionality. However, the data often has an underlying low-dimensional structure. For example, when optimizing via gradient descent, the gradients often lie in or near a low-dimensional subspace. If that low-dimensional structure can be identified, then we can avoid paying (in terms of privacy or accuracy) for the high ambient dimension. We present differentially private algorithms that take input data sampled from a low-dimensional linear subspace (possibly with a small amount of error) and output that subspace (or an approximation to it). These algorithms can serve as a pre-processing step for other procedures.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/07/2020

Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identification

Differentially private SGD (DP-SGD) is one of the most popular methods f...
research
02/18/2012

Robust computation of linear models by convex relaxation

Consider a dataset of vector-valued observations that consists of noisy ...
research
05/22/2023

Representing Input Transformations by Low-Dimensional Parameter Subspaces

Deep models lack robustness to simple input transformations such as rota...
research
08/14/2020

Dimension Independence in Unconstrained Private ERM via Adaptive Preconditioning

In this paper we revisit the problem of private empirical risk minimziat...
research
06/07/2022

Few-Shot Learning by Dimensionality Reduction in Gradient Space

We introduce SubGD, a novel few-shot learning method which is based on t...
research
08/12/2021

Probabilistic methods for approximate archetypal analysis

Archetypal analysis is an unsupervised learning method for exploratory d...
research
09/02/2021

dbcsp: User-friendly R package for Distance-Based Common Spacial Patterns

Common Spacial Patterns (CSP) is a widely used method to analyse electro...

Please sign up or login with your details

Forgot password? Click here to reset