Equivariant Differentially Private Deep Learning

01/30/2023
by   Florian A. Hölzl, et al.
0

The formal privacy guarantee provided by Differential Privacy (DP) bounds the leakage of sensitive information from deep learning models. In practice, however, this comes at a severe computation and accuracy cost. The recently established state of the art (SOTA) results in image classification under DP are due to the use of heavy data augmentation and large batch sizes, leading to a drastically increased computation overhead. In this work, we propose to use more efficient models with improved feature quality by introducing steerable equivariant convolutional networks for DP training. We demonstrate that our models are able to outperform the current SOTA performance on CIFAR-10 by up to 9% across different ε-values while reducing the number of model parameters by a factor of 35 and decreasing the computation time by more than 90 %. Our results are a large step towards efficient model architectures that make optimal use of their parameters and bridge the privacy-utility gap between private and non-private deep learning for computer vision.

READ FULL TEXT

page 8

page 12

research
09/09/2022

Bridging the Gap: Differentially Private Equivariant Deep Learning for Medical Image Analysis

Machine learning with formal privacy-preserving techniques like Differen...
research
05/06/2022

Large Scale Transfer Learning for Differentially Private Image Classification

Differential Privacy (DP) provides a formal framework for training machi...
research
05/28/2023

Training Private Models That Know What They Don't Know

Training reliable deep learning models which avoid making overconfident ...
research
05/21/2022

Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy

Large convolutional neural networks (CNN) can be difficult to train in t...
research
11/24/2022

Differentially Private Image Classification from Features

Leveraging transfer learning has recently been shown to be an effective ...
research
11/03/2020

Similarity-Based Clustering for Enhancing Image Classification Architectures

Convolutional networks are at the center of best in class computer visio...
research
02/24/2017

Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning

Deep Learning has recently become hugely popular in machine learning, pr...

Please sign up or login with your details

Forgot password? Click here to reset