research
∙
08/17/2023
Hitting the High-Dimensional Notes: An ODE for SGD learning dynamics on GLMs and multi-index models
We analyze the dynamics of streaming stochastic gradient descent (SGD) i...
share
research
∙
08/01/2023
Differentially Private Linear Regression with Linked Data
There has been increasing demand for establishing privacy-preserving met...
research
∙
07/03/2023
Fitting an ellipsoid to a quadratic number of random points
We consider the problem (P) of fitting n standard Gaussian random vector...
research
∙
04/13/2023
High-dimensional limit of one-pass SGD on least squares
We give a description of the high-dimensional limit of one-pass single-b...
research
∙
06/15/2022
Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensions
Stochastic gradient descent (SGD) is a pillar of modern machine learning...
research
∙
06/02/2022
Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions
We analyze the dynamics of large batch stochastic gradient descent with ...
research
∙
05/14/2022
Homogenization of SGD in high-dimensions: Exact dynamics and generalization properties
We develop a stochastic differential equation, called homogenized SGD, f...
research
∙
06/07/2021
Dynamics of Stochastic Momentum Methods on Large-scale, Quadratic Models
We analyze a class of stochastic gradient algorithms with momentum on a ...
research
∙
02/08/2021
SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize Criticality
We propose a new framework, inspired by random matrix theory, for analyz...
research
∙
07/01/2020
Universality for the conjugate gradient and MINRES algorithms on sample covariance matrices
We present a probabilistic analysis of two Krylov subspace methods for s...
research
∙
01/30/2018