research
∙
08/21/2023
Unlocking Accuracy and Fairness in Differentially Private Image Classification
Privacy-preserving machine learning aims to train models on private data...
share
research
∙
07/21/2023
On the Universality of Linear Recurrences Followed by Nonlinear Projections
In this note (work in progress towards a full-length paper) we show that...
research
∙
03/11/2023
Resurrecting Recurrent Neural Networks for Long Sequences
Recurrent Neural Networks (RNNs) offer fast inference on long sequences ...
research
∙
02/27/2023
Differentially Private Diffusion Models Generate Useful Synthetic Images
The ability to generate privacy-preserving synthetic versions of sensiti...
research
∙
04/28/2022
Unlocking High-Accuracy Differentially Private Image Classification through Scale
Differential Privacy (DP) provides a formal privacy guarantee preventing...
research
∙
03/07/2022
Regularising for invariance to data augmentation improves supervised learning
Data augmentation is used in machine learning to make the classifier inv...
research
∙
05/31/2021
A study on the plasticity of neural networks
One aim shared by multiple settings, such as continual learning or trans...
research
∙
05/27/2021
Drawing Multiple Augmentation Samples Per Image During Training Efficiently Decreases Test Error
In computer vision, it is standard practice to draw a single sample from...
research
∙
02/11/2021
High-Performance Large-Scale Image Recognition Without Normalization
Batch normalization is a key component of most image classification mode...
research
∙
01/28/2021
On the Origin of Implicit Regularization in Stochastic Gradient Descent
For infinitesimal learning rates, stochastic gradient descent (SGD) foll...
research
∙
01/21/2021
Characterizing signal propagation to close the performance gap in unnormalized ResNets
Batch Normalization is a key component in almost all state-of-the-art im...
research
∙
10/20/2020
BYOL works even without batch statistics
Bootstrap Your Own Latent (BYOL) is a self-supervised learning approach ...
research
∙
06/26/2020
On the Generalization Benefit of Noise in Stochastic Gradient Descent
It has long been argued that minibatch stochastic gradient descent can g...
research
∙
02/24/2020
Batch Normalization Biases Deep Residual Networks Towards Shallow Paths
Batch normalization has multiple benefits. It improves the conditioning ...
research
∙
07/04/2019
Adversarial Robustness through Local Linearization
Adversarial training is an effective methodology for training deep neura...
research
∙
04/15/2019
The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent
The goal of this paper is to study why stochastic gradient descent (SGD)...
research
∙
07/18/2018
Convergence guarantees for RMSProp and ADAM in non-convex optimization and their comparison to Nesterov acceleration on autoencoders
RMSProp and ADAM continue to be extremely popular algorithms for trainin...
research
∙
04/19/2018
Tipping Points for Norm Change in Human Cultures
Humans interact with each other on a daily basis by developing and maint...
research
∙
06/07/2017
Training Quantized Nets: A Deeper Understanding
Currently, deep neural networks are deployed on low-power portable devic...
research
∙
04/16/2017
Understanding Norm Change: An Evolutionary Game-Theoretic Approach (Extended Version)
Human societies around the world interact with each other by developing ...
research
∙
01/09/2017
Son of Zorn's Lemma: Targeted Style Transfer Using Instance-aware Semantic Segmentation
Style transfer is an important task in which the style of a source image...
research
∙
10/18/2016
Big Batch SGD: Automated Inference using Adaptive Batch Sizes
Classical stochastic gradient methods for optimization rely on noisy gra...
research
∙
12/09/2015
Efficient Distributed SGD with Variance Reduction
Stochastic Gradient Descent (SGD) has become one of the most popular opt...
research
∙
12/05/2015
Variance Reduction for Distributed Stochastic Gradient Descent
Variance reduction (VR) methods boost the performance of stochastic grad...
research
∙
10/15/2015
Layer-Specific Adaptive Learning Rates for Deep Networks
The increasing complexity of deep learning architectures is resulting in...
research
∙
02/27/2015