Sashank J Reddi

is this you? claim profile

0 followers
Google

Distilling Double Descent

Distillation is the technique of training a "student" model based on exa...
02/13/2021 ∙ by Andrew Cotter, et al. ∙ 0 ∙ share

read it
Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning

Federated learning is a challenging optimization problem due to the hete...
08/08/2020 ∙ by Sai Praneeth Karimireddy, et al. ∙ 0 ∙ share

read it
O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformers

Transformer networks use pairwise attention to compute contextual embedd...
06/08/2020 ∙ by Chulhee Yun, et al. ∙ 5 ∙ share

read it
Why distillation helps: a statistical perspective

Knowledge distillation is a technique for improving the performance of a...
05/21/2020 ∙ by Aditya Krishna Menon, et al. ∙ 41 ∙ share

read it
Doubly-stochastic mining for heterogeneous retrieval

Modern retrieval problems are characterised by training sets with potent...
04/23/2020 ∙ by Ankit Singh Rawat, et al. ∙ 6 ∙ share

read it
Low-Rank Bottleneck in Multi-head Attention Models

Attention based Transformer architecture has enabled significant advance...
02/17/2020 ∙ by Srinadh Bhojanapalli, et al. ∙ 9 ∙ share

read it
Are Transformers universal approximators of sequence-to-sequence functions?

Despite the widespread adoption of Transformer models for NLP tasks, the...
12/20/2019 ∙ by Chulhee Yun, et al. ∙ 24 ∙ share

read it
Why ADAM Beats SGD for Attention Models

While stochastic gradient descent (SGD) is still the de facto algorithm ...
12/06/2019 ∙ by Jingzhao Zhang, et al. ∙ 0 ∙ share

read it
SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning

Federated learning is a key scenario in modern large-scale machine learn...
10/14/2019 ∙ by Sai Praneeth Karimireddy, et al. ∙ 7 ∙ share

read it
AdaCliP: Adaptive Clipping for Private SGD

Privacy preserving machine learning algorithms are crucial for learning ...
08/20/2019 ∙ by Venkatadheeraj Pichapati, et al. ∙ 1 ∙ share

read it
On the Convergence of Adam and Beyond

Several recently proposed stochastic optimization methods that have been...
04/19/2019 ∙ by Sashank J Reddi, et al. ∙ 30 ∙ share

read it
Escaping Saddle Points with Adaptive Gradient Methods

Adaptive methods such as Adam and RMSProp are widely used in deep learni...
01/26/2019 ∙ by Matthew Staib, et al. ∙ 0 ∙ share

read it
Stochastic Negative Mining for Learning with Large Output Spaces

We consider the problem of retrieving the most relevant labels for a giv...
10/16/2018 ∙ by Sashank J Reddi, et al. ∙ 0 ∙ share

read it
A Generic Approach for Escaping Saddle points

A central challenge to using first-order methods for optimizing nonconve...
09/05/2017 ∙ by Sashank J Reddi, et al. ∙ 0 ∙ share

read it
AIDE: Fast and Communication Efficient Distributed Optimization

In this paper, we present two new communication-efficient methods for di...
08/24/2016 ∙ by Sashank J Reddi, et al. ∙ 0 ∙ share

read it
Stochastic Frank-Wolfe Methods for Nonconvex Optimization

We study Frank-Wolfe methods for nonconvex stochastic and finite-sum opt...
07/27/2016 ∙ by Sashank J Reddi, et al. ∙ 0 ∙ share

read it
Fast Stochastic Methods for Nonsmooth Nonconvex Optimization

We analyze stochastic algorithms for optimizing nonconvex, nonsmooth fin...
05/23/2016 ∙ by Sashank J Reddi, et al. ∙ 0 ∙ share

read it
Stochastic Variance Reduction for Nonconvex Optimization

We study nonconvex finite-sum problems and analyze stochastic variance r...
03/19/2016 ∙ by Sashank J Reddi, et al. ∙ 0 ∙ share

read it
Fast Incremental Method for Nonconvex Optimization

We analyze a fast incremental aggregated gradient method for optimizing ...
03/19/2016 ∙ by Sashank J Reddi, et al. ∙ 0 ∙ share

read it
Adaptivity and Computation-Statistics Tradeoffs for Kernel and Distance based High Dimensional Two Sample Testing

Nonparametric two sample testing is a decision theoretic problem that in...
08/04/2015 ∙ by Aaditya Ramdas, et al. ∙ 0 ∙ share

read it
On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants

We study optimization algorithms based on variance reduction for stochas...
06/23/2015 ∙ by Sashank J Reddi, et al. ∙ 0 ∙ share

read it
On the High-dimensional Power of Linear-time Kernel Two-Sample Testing under Mean-difference Alternatives

Nonparametric two sample testing deals with the question of consistently...
11/23/2014 ∙ by Aaditya Ramdas, et al. ∙ 0 ∙ share

read it
A Maximum Likelihood Approach For Selecting Sets of Alternatives

We consider the problem of selecting a subset of alternatives given nois...
10/16/2012 ∙ by Ariel D. Procaccia, et al. ∙ 0 ∙ share

read it

Sashank J Reddi

Featured Co-authors

Ruslan Salakhutdinov

Francis Bach

Peter Richtarik

Barnabas Poczos

Larry Wasserman

Martin Jaggi

Suvrit Sra

Aarti Singh

Aaditya Ramdas

Sanjiv Kumar

Manzil Zaheer

Distilling Double Descent

Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning

O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformers

Why distillation helps: a statistical perspective

Doubly-stochastic mining for heterogeneous retrieval

Low-Rank Bottleneck in Multi-head Attention Models

Are Transformers universal approximators of sequence-to-sequence functions?

Why ADAM Beats SGD for Attention Models

SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning

AdaCliP: Adaptive Clipping for Private SGD

On the Convergence of Adam and Beyond

Escaping Saddle Points with Adaptive Gradient Methods

Stochastic Negative Mining for Learning with Large Output Spaces

A Generic Approach for Escaping Saddle points

AIDE: Fast and Communication Efficient Distributed Optimization

Stochastic Frank-Wolfe Methods for Nonconvex Optimization

Fast Stochastic Methods for Nonsmooth Nonconvex Optimization

Stochastic Variance Reduction for Nonconvex Optimization

Fast Incremental Method for Nonconvex Optimization

Adaptivity and Computation-Statistics Tradeoffs for Kernel and Distance based High Dimensional Two Sample Testing

On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants

On the High-dimensional Power of Linear-time Kernel Two-Sample Testing under Mean-difference Alternatives

A Maximum Likelihood Approach For Selecting Sets of Alternatives