research
∙
08/31/2023
Transformers as Support Vector Machines
Since its inception in "Attention Is All You Need", transformer architec...
share
research
∙
08/03/2023
Memory capacity of two layer neural networks with smooth activations
Determining the memory capacity of two-layer neural networks with m hidd...
research
∙
06/13/2023
Supervised-Contrastive Loss Learns Orthogonal Frames and Batching Matters
Supervised contrastive loss (SCL) is a competitive and often superior al...
research
∙
06/06/2023
On the Role of Attention in Prompt-tuning
Prompt-tuning is an emerging strategy to adapt large language models (LL...
research
∙
06/03/2023
Memorization Capacity of Multi-Head Attention in Transformers
In this paper, we investigate the memorization capabilities of multi-hea...
research
∙
05/30/2023
BiSLS/SPS: Auto-tune Step Sizes for Stable Bi-level Optimization
The popularity of bi-level optimization (BO) in deep learning has spurre...
research
∙
05/22/2023
Fast Convergence in Learning Two-Layer Neural Networks with Separable Data
Normalized gradient descent has shown substantial success in speeding up...
research
∙
04/02/2023
Fast Convergence of Random Reshuffling under Over-Parameterization and the Polyak-Łojasiewicz Condition
Modern machine learning models are often over-parameterized and as a res...
research
∙
03/14/2023
On the Implicit Geometry of Cross-Entropy Parameterizations for Label-Imbalanced Data
Various logit-adjusted parameterizations of the cross-entropy (CE) loss ...
research
∙
02/18/2023
Generalization and Stability of Interpolating Neural Networks with Minimal Width
We investigate the generalization and optimization of k-homogeneous shal...
research
∙
09/15/2022
Decentralized Learning with Separable Data: Generalization and Fast Algorithms
Decentralized learning offers privacy and communication efficiency when ...
research
∙
08/10/2022
Imbalance Trouble: Revisiting Neural-Collapse Geometry
Neural Collapse refers to the remarkable structural properties character...
research
∙
06/25/2022
On how to avoid exacerbating spurious correlations when models are overparameterized
Overparameterized models fail to generalize well in the presence of data...
research
∙
05/25/2022
Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently
Driven by the empirical success and wide use of deep neural networks, un...
research
∙
05/12/2022
Multi-Environment Meta-Learning in Stochastic Linear Bandits
In this work we investigate meta-learning (or learning-to-learn) approac...
research
∙
05/04/2022
FEDNEST: Federated Bilevel, Minimax, and Compositional Optimization
Standard federated optimization methods successfully apply to stochastic...
research
∙
01/04/2022
AutoBalance: Optimized Loss Functions for Imbalanced Data
Imbalanced datasets are commonplace in modern machine learning problems....
research
∙
09/20/2021
Sharp global convergence guarantees for iterative nonconvex optimization: A Gaussian process perspective
We consider a general class of regression models with normally distribut...
research
∙
06/21/2021
Benign Overfitting in Multiclass Classification: All Roads Lead to Interpolation
The growing literature on "benign overfitting" in overparameterized mode...
research
∙
06/11/2021
Safe Reinforcement Learning with Linear Function Approximation
Safety in reinforcement learning has become increasingly important in re...
research
∙
03/21/2021
UCB-based Algorithms for Multinomial Logistic Regression Bandits
Out of the rich family of generalized linear bandits, perhaps the most w...
research
∙
03/02/2021
Label-Imbalanced and Group-Sensitive Classification under Overparameterization
Label-imbalanced and group-sensitive classification seeks to appropriate...
research
∙
12/16/2020
Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks
Deep networks are typically trained with many more parameters than the s...
research
∙
12/01/2020
Decentralized Multi-Agent Linear Bandits with Safety Constraints
We study decentralized stochastic linear bandits, where a network of N a...
research
∙
11/18/2020
Benign Overfitting in Binary Classification of Gaussian Mixtures
Deep neural networks generalize well despite being exceedingly overparam...
research
∙
11/16/2020
Theoretical Insights Into Multiclass Classification: A High-dimensional Asymptotic View
Contemporary machine learning applications often involve classification ...
research
∙
10/26/2020
Asymptotic Behavior of Adversarial Training in Binary Classification
It is widely known that several machine learning models are susceptible ...
research
∙
09/30/2020
Stage-wise Conservative Linear Bandits
We study stage-wise conservative linear stochastic bandits: an instance ...
research
∙
08/07/2020
Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models
We study the problem of recovering an unknown signal x given measurement...
research
∙
06/19/2020
Exploring Weight Importance and Hessian Bias in Model Pruning
Model pruning is an essential procedure for building compact and computa...
research
∙
06/16/2020
Fundamental Limits of Ridge-Regularized Empirical Risk Minimization in High Dimensions
Empirical Risk Minimization (ERM) algorithms are widely used in a variet...
research
∙
05/05/2020
Regret Bounds for Safe Gaussian Process Bandit Optimization
Many applications require a learner to make sequential decisions given u...
research
∙
02/17/2020
Sharp Asymptotics and Optimal Performance for Inference in Binary Models
We study convex empirical risk minimization for high-dimensional inferen...
research
∙
01/30/2020
Analytic Study of Double Descent in Binary Classification: The Impact of Loss
Extensive empirical evidence reveals that, for a wide range of different...
research
∙
11/13/2019
A Model of Double Descent for High-dimensional Binary Linear Classification
We consider a model for logistic regression where only a subset of featu...
research
∙
11/06/2019
Safe Linear Thompson Sampling
The design and performance analysis of bandit algorithms in the presence...
research
∙
08/16/2019
Linear Stochastic Bandits Under Safety Constraints
Bandit algorithms have various application in safety-critical systems, w...
research
∙
08/12/2019
Sharp Guarantees for Solving Random Equations with One-Bit Information
We study the performance of a wide class of convex optimization-based es...
research
∙
03/10/2019
A simple bound on the BER of the MAP decoder for massive MIMO systems
The deployment of massive MIMO systems has revived much of the interest ...
research
∙
07/18/2018
The Generalized Lasso for Sub-gaussian Measurements with Dithered Quantization
In the problem of structured signal recovery from high-dimensional linea...
research
∙
05/24/2018
Phase Retrieval via Polytope Optimization: Geometry, Phase Transitions, and New Algorithms
We study algorithms for solving quadratic systems of equations based on ...
research
∙
02/10/2018
Revealing hidden scenes by photon-efficient occlusion-based opportunistic active imaging
The ability to see around corners, i.e., recover details of a hidden sce...
research
∙
12/11/2017
The PhaseLift for Non-quadratic Gaussian Measurements
We study the problem of recovering a structured signal x_0 from high-dim...
research
∙
11/30/2017