research
∙
12/28/2022
On Implicit Bias in Overparameterized Bilevel Optimization
Many problems in machine learning involve bilevel optimization (BLO), in...
share
research
∙
12/08/2022
A Novel Stochastic Gradient Descent Algorithm for Learning Principal Subspaces
Many machine learning problems encode their data as a matrix with a poss...
research
∙
11/09/2022
Extragradient with Positive Momentum is Optimal for Games with Cross-Shaped Jacobian Spectrum
The extragradient method has recently gained increasing attention, due t...
research
∙
10/10/2022
Second-order regression models exhibit progressive sharpening to the edge of stability
Recent studies of gradient descent with large step sizes have shown that...
research
∙
09/27/2022
The Curse of Unrolling: Rate of Differentiating Through Optimization
Computing the Jacobian of the solution of an optimization problem is a c...
research
∙
06/20/2022
Only Tails Matter: Average-Case Universality and Robustness in the Convex Regime
The recently developed average-case analysis of optimization methods all...
research
∙
02/24/2022
Cutting Some Slack for SGD with Adaptive Polyak Stepsizes
Tuning the step size of stochastic gradient descent is tedious and error...
research
∙
01/13/2022
GradMax: Growing Neural Networks using Gradient Information
The architecture and the parameters of neural networks are often optimiz...
research
∙
05/31/2021
Efficient and Modular Implicit Differentiation
Automatic differentiation (autodiff) has revolutionized machine learning...
research
∙
05/19/2021
Boosting Variational Inference With Locally Adaptive Step-Sizes
Variational Inference makes a trade-off between the capacity of the vari...
research
∙
02/08/2021
SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize Criticality
We propose a new framework, inspired by random matrix theory, for analyz...
research
∙
10/05/2020
Average-case Acceleration for Bilinear Games and Normal Matrices
Advances in generative modeling and adversarial learning have given rise...
research
∙
06/08/2020
Halting Time is Predictable for Large Models: A Universality Property and Average-case Analysis
Average-case analysis computes the complexity of an algorithm averaged o...
research
∙
02/12/2020
Average-case Acceleration Through Spectral Density Estimation
We develop a framework for designing optimal quadratic optimization meth...
research
∙
10/08/2019
A Test for Shared Patterns in Cross-modal Brain Activation Analysis
Determining the extent to which different cognitive modalities (understo...
research
∙
07/23/2019
SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
SciPy is an open source scientific computing library for the Python prog...
research
∙
06/25/2019
The Difficulty of Training Sparse Neural Networks
We investigate the difficulties of training sparse neural networks and m...
research
∙
06/18/2019
Information matrices and generalization
This work revisits the use of information criteria to characterize the g...
research
∙
04/09/2018
Frank-Wolfe Splitting via Augmented Lagrangian Method
Minimizing a function over an intersection of convex sets is an importan...
research
∙
04/06/2018
Adaptive Three Operator Splitting
We propose and analyze a novel adaptive step size variant of the Davis-Y...
research
∙
03/20/2018
Frank-Wolfe with Subsampling Oracle
We analyze two novel randomized variants of the Frank-Wolfe (FW) or cond...
research
∙
07/20/2017
Breaking the Nonsmooth Barrier: A Scalable Parallel Method for Composite Optimization
Due to their simplicity and excellent performance, parallel asynchronous...
research
∙
10/25/2016
On the convergence rate of the three operator splitting scheme
The three operator splitting scheme was recently proposed by [Davis and ...
research
∙
06/15/2016
ASAGA: Asynchronous Parallel SAGA
We describe ASAGA, an asynchronous parallel version of the incremental g...
research
∙
02/07/2016
Hyperparameter optimization with approximate gradient
Most models in machine learning contain at least one hyperparameter to c...
research
∙
12/12/2014
Machine Learning for Neuroimaging with Scikit-Learn
Statistical machine learning methods are increasingly used for neuroimag...
research
∙
08/10/2013
Second order scattering descriptors predict fMRI activity due to visual textures
Second layer scattering descriptors are known to provide good classifica...
research
∙
07/16/2012
Learning to rank from medical imaging data
Medical images can be used to predict a clinical score coding for the se...
research
∙
07/15/2012