Ankit Singh Rawat

is this you? claim profile

0 followers
Google ∙ MIT

Modifying Memories in Transformer Models

Large Transformer models have achieved impressive performance in many na...
12/01/2020 ∙ by Chen Zhu, et al. ∙ 0 ∙ share

read it
O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformers

Transformer networks use pairwise attention to compute contextual embedd...
06/08/2020 ∙ by Chulhee Yun, et al. ∙ 5 ∙ share

read it
Why distillation helps: a statistical perspective

Knowledge distillation is a technique for improving the performance of a...
05/21/2020 ∙ by Aditya Krishna Menon, et al. ∙ 41 ∙ share

read it
Doubly-stochastic mining for heterogeneous retrieval

Modern retrieval problems are characterised by training sets with potent...
04/23/2020 ∙ by Ankit Singh Rawat, et al. ∙ 6 ∙ share

read it
Federated Learning with Only Positive Labels

We consider learning a multi-class classification model in the federated...
04/21/2020 ∙ by Felix X. Yu, et al. ∙ 9 ∙ share

read it
Robust Large-Margin Learning in Hyperbolic Space

Recently, there has been a surge of interest in representation learning ...
04/11/2020 ∙ by Melanie Weber, et al. ∙ 12 ∙ share

read it
Reliable Distributed Clustering with Redundant Data Assignment

In this paper, we present distributed generalized clustering algorithms ...
02/20/2020 ∙ by Venkata Gandikota, et al. ∙ 0 ∙ share

read it
Low-Rank Bottleneck in Multi-head Attention Models

Attention based Transformer architecture has enabled significant advance...
02/17/2020 ∙ by Srinadh Bhojanapalli, et al. ∙ 9 ∙ share

read it
Achieving Multi-Port Memory Performance on Single-Port Memory with Coding Techniques

Many performance critical systems today must rely on performance enhance...
01/27/2020 ∙ by Hardik Jain, et al. ∙ 0 ∙ share

read it
Are Transformers universal approximators of sequence-to-sequence functions?

Despite the widespread adoption of Transformer models for NLP tasks, the...
12/20/2019 ∙ by Chulhee Yun, et al. ∙ 24 ∙ share

read it
Sampled Softmax with Random Fourier Features

The computational cost of training with softmax cross entropy loss grows...
07/24/2019 ∙ by Ankit Singh Rawat, et al. ∙ 5 ∙ share

read it
The Generalized Lasso for Sub-gaussian Measurements with Dithered Quantization

In the problem of structured signal recovery from high-dimensional linea...
07/18/2018 ∙ by Christos Thrampoulidis, et al. ∙ 0 ∙ share

read it
Robust Gradient Descent via Moment Encoding with LDPC Codes

This paper considers the problem of implementing large-scale gradient de...
05/22/2018 ∙ by Raj Kumar Maity, et al. ∙ 0 ∙ share

read it
Representation Learning and Recovery in the ReLU Model

Rectified linear units, or ReLUs, have become the preferred activation f...
03/12/2018 ∙ by Arya Mazumdar, et al. ∙ 0 ∙ share

read it
The PhaseLift for Non-quadratic Gaussian Measurements

We study the problem of recovering a structured signal x_0 from high-dim...
12/11/2017 ∙ by Christos Thrampoulidis, et al. ∙ 0 ∙ share

read it
MDS Code Constructions with Small Sub-packetization and Near-optimal Repair Bandwidth

This paper addresses the problem of constructing MDS codes that enable e...
09/24/2017 ∙ by Ankit Singh Rawat, et al. ∙ 0 ∙ share

read it

Ankit Singh Rawat

Featured Co-authors

Sanjiv Kumar

Manzil Zaheer

Venkatesan Guruswami

Ananda Theertha Suresh

Arya Mazumdar

Sriram Vishwanath

Chen Zhu

Christos Thrampoulidis

Sashank J Reddi

Srinadh Bhojanapalli

Aditya Krishna Menon

Modifying Memories in Transformer Models

O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformers

Why distillation helps: a statistical perspective

Doubly-stochastic mining for heterogeneous retrieval

Federated Learning with Only Positive Labels

Robust Large-Margin Learning in Hyperbolic Space

Reliable Distributed Clustering with Redundant Data Assignment

Low-Rank Bottleneck in Multi-head Attention Models

Achieving Multi-Port Memory Performance on Single-Port Memory with Coding Techniques

Are Transformers universal approximators of sequence-to-sequence functions?

Sampled Softmax with Random Fourier Features

The Generalized Lasso for Sub-gaussian Measurements with Dithered Quantization

Robust Gradient Descent via Moment Encoding with LDPC Codes

Representation Learning and Recovery in the ReLU Model

The PhaseLift for Non-quadratic Gaussian Measurements

MDS Code Constructions with Small Sub-packetization and Near-optimal Repair Bandwidth