research
∙
07/20/2023
Cluster-aware Semi-supervised Learning: Relational Knowledge Distillation Provably Learns Clustering
Despite the empirical success and practical significance of (relational)...
share
research
∙
05/11/2023
Convergence of Alternating Gradient Descent for Matrix Factorization
We consider alternating gradient descent (AGD) with fixed step size η > ...
research
∙
05/09/2023
Robust Implicit Regularization via Weight Normalization
Overparameterized models may have many interpolating solutions; implicit...
research
∙
10/04/2022
AdaWAC: Adaptively Weighted Augmentation Consistency Regularization for Volumetric Medical Image Segmentation
Sample reweighting is an effective strategy for learning from training d...
research
∙
06/15/2022
On the fast convergence of minibatch heavy ball momentum
Simple stochastic momentum methods are widely used in machine learning o...
research
∙
05/19/2022
How catastrophic can catastrophic forgetting be in linear regression?
To better understand catastrophic forgetting, we study fitting an overpa...
research
∙
05/16/2022
An Exponentially Increasing Step-size for Parameter Estimation in Statistical Models
Using gradient descent (GD) with fixed or decaying step-size is standard...
research
∙
04/22/2022
Scalable symmetric Tucker tensor decomposition
We study the best low-rank Tucker decomposition of symmetric tensors, ad...
research
∙
04/14/2022
Concentration of Random Feature Matrices in High-Dimensions
The spectra of random feature matrices provide essential information on ...
research
∙
03/01/2022
Side-effects of Learning from Low Dimensional Data Embedded in an Euclidean Space
The low dimensional manifold hypothesis posits that the data found in ma...
research
∙
02/24/2022
Sample Efficiency of Data Augmentation Consistency Regularization
Data augmentation is popular in the training of large neural networks; c...
research
∙
02/11/2022
The Power of Adaptivity in SGD: Self-Tuning Step Sizes with Unbounded Gradients and Affine Variance
We study convergence rates of AdaGrad-Norm as an exemplar of adaptive st...
research
∙
12/07/2021
SHRIMP: Sparser Random Feature Models via Iterative Magnitude Pruning
Sparse shrunk additive models and sparse random feature models have been...
research
∙
09/20/2021
Learning to Forecast Dynamical Systems from Streaming Data
Kernel analog forecasting (KAF) is a powerful methodology for data-drive...
research
∙
09/17/2021
AdaLoss: A computationally-efficient and provably convergent adaptive gradient method
We propose a computationally-friendly adaptive learning rate schedule, "...
research
∙
08/17/2021
Arbitrary-length analogs to de Bruijn sequences
Let α be a length-L cyclic sequence of characters from a size-K alphabet...
research
∙
06/28/2021
Bootstrapping the error of Oja's Algorithm
We consider the problem of quantifying uncertainty for the estimation er...
research
∙
06/24/2021
Johnson-Lindenstrauss Embeddings with Kronecker Structure
We prove the Johnson-Lindenstrauss property for matrices Φ D_ξ where Φ h...
research
∙
03/04/2021
Function Approximation via Sparse Random Features
Random feature methods have been successful in various machine learning ...
research
∙
02/06/2021
Streaming k-PCA: Efficient guarantees for Oja's algorithm, beyond rank-one updates
We analyze Oja's algorithm for streaming k-PCA and prove that it achieve...
research
∙
06/15/2020
Weighted Optimization: better generalization by smoother interpolation
We provide a rigorous analysis of how implicit bias towards smooth inter...
research
∙
11/18/2019
Implicit Regularization of Normalization Methods
Normalization methods such as batch normalization are commonly used in o...
research
∙
09/11/2019
Faster Johnson-Lindenstrauss Transforms via Kronecker Products
The Kronecker product is an important matrix operation with a wide range...
research
∙
08/28/2019
Linear Convergence of Adaptive Stochastic Gradient Descent
We prove that the norm version of the adaptive stochastic gradient metho...
research
∙
07/26/2019
Bias of Homotopic Gradient Descent for the Hinge Loss
Gradient descent is a simple and widely used optimization method for mac...
research
∙
05/28/2019
AdaOja: Adaptive Learning Rates for Streaming PCA
Oja's algorithm has been the cornerstone of streaming methods in Princip...
research
∙
02/19/2019
Global Convergence of Adaptive Gradient Methods for An Over-parameterized Neural Network
Adaptive gradient methods like AdaGrad are widely used in optimizing neu...
research
∙
11/25/2018
Recovery guarantees for polynomial approximation from dependent data with outliers
Learning non-linear systems from noisy, limited, and/or dependent data i...
research
∙
09/18/2018
Compressed sensing with a jackknife and a bootstrap
Compressed sensing proposes to reconstruct more degrees of freedom in a ...
research
∙
06/05/2018
AdaGrad stepsizes: Sharp convergence over nonconvex landscapes, from any initialization
Adaptive gradient methods such as AdaGrad and its variants update the st...
research
∙
05/10/2018
Extracting structured dynamical systems using sparse optimization with very few samples
Learning governing equations allows for deeper understanding of the stru...
research
∙
03/28/2018
Greedy Variance Estimation for the LASSO
Recent results have proven the minimax optimality of LASSO and related a...
research
∙
03/07/2018
WNGrad: Learn the Learning Rate in Gradient Descent
Adjusting the learning rate schedule in stochastic gradient methods is a...
research
∙
10/17/2016
A polynomial-time relaxation of the Gromov-Hausdorff distance
The Gromov-Hausdorff distance provides a metric on the set of isometry c...
research
∙
02/22/2016
Clustering subgaussian mixtures by semidefinite programming
We introduce a model-free relax-and-round algorithm for k-means clusteri...
research
∙
06/25/2015
The local convexity of solving systems of quadratic equations
This paper considers the recovery of a rank r positive semidefinite matr...
research
∙
04/28/2014
One-bit compressive sensing with norm estimation
Consider the recovery of an unknown signal x from quantized linear measu...
research
∙
10/21/2013
Stochastic Gradient Descent, Weighted Sampling, and the Randomized Kaczmarz algorithm
We obtain an improved finite-sample guarantee on the linear convergence ...
research
∙
09/12/2013
Recovery guarantees for exemplar-based clustering
For a certain class of distributions, we prove that the linear programmi...
research
∙
06/12/2013
Completing Any Low-rank Matrix, Provably
Matrix completion, i.e., the exact and provable recovery of a low-rank m...
research
∙
10/11/2012
Near-optimal compressed sensing guarantees for total variation minimization
Consider the problem of reconstructing a multidimensional signal from an...
research
∙
10/08/2012