research
∙
06/01/2023
A Uniform Confidence Phenomenon in Deep Learning and its Implications for Calibration
Despite the impressive generalization capabilities of deep neural networ...
share
research
∙
05/18/2023
Smoothing the Landscape Boosts the Signal for SGD: Optimal Sample Complexity for Learning Single Index Models
We focus on the task of learning a single index model σ(w^⋆· x) with res...
research
∙
04/03/2023
Depth Separation with Multilayer Mean-Field Networks
Depth separation – why a deeper network is more powerful than a shallowe...
research
∙
03/14/2023
Do Transformers Parse while Predicting the Masked Word?
Pre-trained language models have been shown to encode linguistic structu...
research
∙
02/24/2023
Hiding Data Helps: On the Benefits of Masking for Sparse Coding
Sparse coding refers to modeling a signal as sparse linear combinations ...
research
∙
02/01/2023
Implicit Regularization Leads to Benign Overfitting for Sparse Linear Regression
In deep learning, often the training process finds an interpolator (a so...
research
∙
10/24/2022
Provably Learning Diverse Features in Multi-View Data with Midpoint Mixup
Mixup is a data augmentation technique that relies on training using ran...
research
∙
10/07/2022
Understanding Edge-of-Stability Training Dynamics with a Minimalist Example
Recently, researchers observed that gradient descent for deep neural net...
research
∙
10/03/2022
Plateau in Monotonic Linear Interpolation – A "Biased" View of Loss Landscape for Deep Networks
Monotonic linear interpolation (MLI) - on the line connecting a random i...
research
∙
05/18/2022
Customizing ML Predictions for Online Algorithms
A popular line of recent research incorporates ML advice in the design o...
research
∙
10/14/2021
Towards Understanding the Data Dependency of Mixup-style Training
In the Mixup training paradigm, a model is trained using convex combinat...
research
∙
09/23/2021
Outlier-Robust Sparse Estimation via Non-Convex Optimization
We explore the connection between outlier-robust high-dimensional statis...
research
∙
06/11/2021
Understanding Deflation Process in Over-parametrized Tensor Decomposition
In this paper we study the training dynamics for gradient flow on over-p...
research
∙
02/04/2021
A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network
While over-parameterization is widely believed to be crucial for the suc...
research
∙
10/22/2020
Beyond Lazy Training for Over-parameterized Tensor Decomposition
Over-parametrization is an important technique in training neural networ...
research
∙
10/08/2020
Dissecting Hessian: Understanding Common Structure of Hessian in Neural Networks
Hessian captures important properties of the deep neural network loss la...
research
∙
09/30/2020
Efficient sampling from the Bingham distribution
We give a algorithm for exact sampling from the Bingham distribution p(x...
research
∙
06/30/2020
Guarantees for Tuning the Step Size using a Learning-to-Learn Approach
Learning-to-learn (using optimization algorithms to learn a new optimize...
research
∙
06/29/2020
Optimization Landscape of Tucker Decomposition
Tucker decomposition is a popular technique for many data analysis and m...
research
∙
06/29/2020
Extracting Latent State Representations with Linear Dynamics from Rich Observations
Recently, many reinforcement learning techniques were shown to have prov...
research
∙
05/12/2020
Energy-Aware DNN Graph Optimization
Unlike existing work in deep neural network (DNN) graphs optimization fo...
research
∙
05/04/2020
High-Dimensional Robust Mean Estimation via Gradient Descent
We study the problem of high-dimensional robust mean estimation in the p...
research
∙
04/16/2020
Spectral Learning on Matrices and Tensors
Spectral methods have been the mainstay in several domains such as machi...
research
∙
11/08/2019
Estimating Normalizing Constants for Log-Concave Distributions: Algorithms and Lower Bounds
Estimating the normalizing constant of an unnormalized probability distr...
research
∙
09/26/2019
Mildly Overparametrized Neural Nets can Memorize Training Data Efficiently
It has been observed zhang2016understanding that deep neural networks ca...
research
∙
06/14/2019
Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets
Mode connectivity is a surprising phenomenon in the loss landscape of de...
research
∙
06/11/2019
Faster Algorithms for High-Dimensional Robust Covariance Estimation
We study the problem of estimating the covariance matrix of a high-dimen...
research
∙
05/01/2019
Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization
Variance reduction techniques like SVRG provide simple and fast algorith...
research
∙
04/29/2019
The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning Rate Procedure
There is a stark disparity between the step size schedules used in pract...
research
∙
02/13/2019
Stochastic Gradient Descent Escapes Saddle Points Efficiently
This paper considers the perturbed stochastic gradient descent algorithm...
research
∙
02/11/2019
A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm
In this note, we derive concentration inequalities for random vectors wi...
research
∙
02/02/2019
Understanding Composition of Word Embeddings via Tensor Decomposition
Word embedding is a powerful tool in natural language processing. In thi...
research
∙
11/29/2018
Simulated Tempering Langevin Monte Carlo II: An Improved Proof using Soft Markov Chain Decomposition
A key task in Bayesian machine learning is sampling from distributions t...
research
∙
11/23/2018
High-Dimensional Robust Mean Estimation in Nearly-Linear Time
We study the fundamental problem of high-dimensional mean estimation in ...
research
∙
10/16/2018
Learning Two-layer Neural Networks with Symmetric Inputs
We give a new algorithm for learning a two-layer neural network under a ...
research
∙
03/28/2018
Non-Convex Matrix Completion Against a Semi-Random Adversary
Matrix completion is a well-studied problem with many machine learning a...
research
∙
03/25/2018
Minimizing Nonconvex Population Risk from Rough Empirical Risk
Population risk---the expectation of the loss over the sampling mechanis...
research
∙
02/14/2018
Stronger generalization bounds for deep nets via a compression approach
Deep nets generalize well despite having more parameters than the number...
research
∙
01/15/2018
Global Convergence of Policy Gradient Methods for Linearized Control Problems
Direct policy gradient methods for reinforcement learning and continuous...
research
∙
11/01/2017
Learning One-hidden-layer Neural Networks with Landscape Design
We consider the problem of learning a one-hidden-layer neural network: w...
research
∙
10/07/2017
Beyond Log-concavity: Provable Guarantees for Sampling Multi-modal Distributions using Simulated Tempering Langevin Monte Carlo
A key task in Bayesian statistics is sampling from distributions that ar...
research
∙
06/18/2017
On the Optimization Landscape of Tensor Decompositions
Non-convex optimization with local search heuristics has been widely use...
research
∙
04/03/2017
No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis
In this paper we develop a new framework that captures the common landsc...
research
∙
03/02/2017
How to Escape Saddle Points Efficiently
This paper shows that a perturbed form of gradient descent converges to ...
research
∙
03/02/2017
Generalization and Equilibrium in Generative Adversarial Nets (GANs)
We show that training of generative adversarial network (GAN) may not ha...
research
∙
12/28/2016
Provable learning of Noisy-or Networks
Many machine learning applications use latent variable models to explain...
research
∙
10/28/2016
Homotopy Analysis for Tensor PCA
Developing efficient and guaranteed nonconvex algorithms has been an imp...
research
∙
09/29/2016
DynIMS: A Dynamic Memory Controller for In-memory Storage on HPC Systems
In order to boost the performance of data-intensive computing on HPC sys...
research
∙
05/27/2016
Provable Algorithms for Inference in Topic Models
Recently, there has been considerable progress on designing algorithms w...
research
∙
05/24/2016