Samuel L. Smith | DeepAI

AI Chat

Featured Co-authors

Yee Whye Teh
122 publications
Quoc V. Le
113 publications
Pushmeet Kohli
93 publications
Razvan Pascanu
87 publications
Jascha Sohl-Dickstein
68 publications
Karen Simonyan
53 publications
Caglar Gulcehre
42 publications
Jasper Snoek
34 publications
Sven Gowal
31 publications
Borja Balle
28 publications
Guodong Zhang
26 publications

∙ 08/21/2023

Unlocking Accuracy and Fairness in Differentially Private Image Classification

Privacy-preserving machine learning aims to train models on private data...

0 Leonard Berrada, et al. ∙

∙ 07/21/2023

On the Universality of Linear Recurrences Followed by Nonlinear Projections

In this note (work in progress towards a full-length paper) we show that...

0 Antonio Orvieto, et al. ∙

∙ 03/11/2023

Resurrecting Recurrent Neural Networks for Long Sequences

Recurrent Neural Networks (RNNs) offer fast inference on long sequences ...

10 Antonio Orvieto, et al. ∙

∙ 02/27/2023

Differentially Private Diffusion Models Generate Useful Synthetic Images

The ability to generate privacy-preserving synthetic versions of sensiti...

3 Sahra Ghalebikesabi, et al. ∙

∙ 02/20/2023

Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation

Skip connections and normalisation layers form two standard architectura...

0 Bobby He, et al. ∙

∙ 04/28/2022

Unlocking High-Accuracy Differentially Private Image Classification through Scale

Differential Privacy (DP) provides a formal privacy guarantee preventing...

0 Soham De, et al. ∙

∙ 05/27/2021

Drawing Multiple Augmentation Samples Per Image During Training Efficiently Decreases Test Error

In computer vision, it is standard practice to draw a single sample from...

0 Stanislav Fort, et al. ∙

∙ 02/11/2021

High-Performance Large-Scale Image Recognition Without Normalization

Batch normalization is a key component of most image classification mode...

0 Andrew Brock, et al. ∙

∙ 01/28/2021

On the Origin of Implicit Regularization in Stochastic Gradient Descent

For infinitesimal learning rates, stochastic gradient descent (SGD) foll...

0 Samuel L. Smith, et al. ∙

∙ 01/21/2021

Characterizing signal propagation to close the performance gap in unnormalized ResNets

Batch Normalization is a key component in almost all state-of-the-art im...

0 Andrew Brock, et al. ∙

∙ 07/31/2020

Cold Posteriors and Aleatoric Uncertainty

Recent work has observed that one can outperform exact inference in Baye...

0 Ben Adlam, et al. ∙

∙ 06/26/2020

On the Generalization Benefit of Noise in Stochastic Gradient Descent

It has long been argued that minibatch stochastic gradient descent can g...

12 Samuel L. Smith, et al. ∙

∙ 02/24/2020

Batch Normalization Biases Deep Residual Networks Towards Shallow Paths

Batch normalization has multiple benefits. It improves the conditioning ...

0 Soham De, et al. ∙

∙ 05/09/2019

The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study

We investigate how the final parameters found by stochastic gradient des...

0 Daniel S. Park, et al. ∙

∙ 06/25/2018

Stochastic natural gradient descent draws posterior samples in function space

Natural gradient descent (NGD) minimises the cost function on a Riemanni...

0 Samuel L. Smith, et al. ∙

∙ 05/09/2018

Decoding Decoders: Finding Optimal Representation Spaces for Unsupervised Similarity Tasks

Experimental evidence indicates that simple models outperform complex de...

0 Vitalii Zhelezniak, et al. ∙

∙ 11/01/2017

Don't Decay the Learning Rate, Increase the Batch Size

It is common practice to decay the learning rate. Here we show one can u...

0 Samuel L. Smith, et al. ∙

∙ 10/17/2017

A Bayesian Perspective on Generalization and Stochastic Gradient Descent

This paper tackles two related questions at the heart of machine learnin...

0 Samuel L. Smith, et al. ∙

∙ 02/13/2017

Offline bilingual word vectors, orthogonal transformations and the inverted softmax

Usually bilingual word vectors are trained "online". Mikolov et al. show...

0 Samuel L. Smith, et al. ∙

∙ 12/27/2016

Monte Carlo Sort for unreliable human comparisons

Algorithms which sort lists of real numbers into ascending order have be...

0 Samuel L. Smith, et al. ∙

Are you a researcher?
Expose your work
to one of the largest
A.I. communities in the world