research
∙
02/15/2023
Continuized Acceleration for Quasar Convex Functions in Non-Convex Optimization
Quasar convexity is a condition that allows some first-order methods to ...
share
research
∙
10/18/2022
Towards Understanding GD with Hard and Conjugate Pseudo-labels for Test-Time Adaptation
We consider a setting that a model needs to adapt to a new domain under ...
research
∙
07/05/2022
Accelerating Hamiltonian Monte Carlo via Chebyshev Integration Time
Hamiltonian Monte Carlo (HMC) is a popular method in sampling. While the...
research
∙
06/22/2022
Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-Łojasiewicz Functions when the Non-Convexity is Averaged-Out
Heavy Ball (HB) nowadays is one of the most popular momentum methods in ...
research
∙
11/22/2021
No-Regret Dynamics in the Fenchel Game: A Unified Framework for Algorithmic Convex Optimization
We develop an algorithmic framework for solving convex optimization prob...
research
∙
06/23/2021
Understanding Modern Techniques in Optimization: Frank-Wolfe, Nesterov's Momentum, and Polyak's Momentum
In the first part of this dissertation research, we develop a modular fr...
research
∙
06/05/2021
Escaping Saddle Points Faster with Stochastic Momentum
Stochastic gradient descent (SGD) with stochastic momentum is popular in...
research
∙
10/04/2020
Understanding How Over-Parametrization Leads to Acceleration: A case of learning a single teacher neuron
Over-parametrization has become a popular technique in deep learning. It...
research
∙
10/04/2020
Provable Acceleration of Neural Net Training via Polyak's Momentum
Incorporating a so-called "momentum" dynamic in gradient descent methods...
research
∙
10/04/2020
Quickly Finding a Benign Region via Heavy Ball Momentum in Non-Convex Optimization
The Heavy Ball Method, proposed by Polyak over five decades ago, is a fi...
research
∙
03/04/2019
Optimistic Adaptive Acceleration for Optimization
We consider a new variant of AMSGrad. AMSGrad RKK18 is a popular adaptiv...
research
∙
11/14/2018
Revisiting Projection-Free Optimization for Strongly Convex Constraint Sets
We revisit the Frank-Wolfe (FW) optimization under strongly convex const...
research
∙
07/27/2018
Acceleration through Optimistic No-Regret Dynamics
We consider the problem of minimizing a smooth convex function by reduci...
research
∙
05/17/2018