research
∙
08/20/2023
A Human-on-the-Loop Optimization Autoformalism Approach for Sustainability
This paper outlines a natural conversational approach to solving persona...
share
research
∙
07/16/2023
MindOpt Tuner: Boost the Performance of Numerical Software by Automatic Parameter Tuning
Numerical software is usually shipped with built-in hyperparameters. By ...
research
∙
06/01/2023
DSGD-CECA: Decentralized SGD with Communication-Optimal Exact Consensus Algorithm
Decentralized Stochastic Gradient Descent (SGD) is an emerging neural ne...
research
∙
05/29/2023
Towards Constituting Mathematical Structures for Learning to Optimize
Learning to Optimize (L2O), a technique that utilizes machine learning t...
research
∙
05/12/2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression
Communication compression is an essential strategy for alleviating commu...
research
∙
11/15/2022
An Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methods
In this paper, we revisit and improve the convergence of policy gradient...
research
∙
11/14/2022
Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimization
Stochastic bilevel optimization, which captures the inherent nested stru...
research
∙
10/19/2022
On Representing Mixed-Integer Linear Programs by Graph Neural Networks
While Mixed-integer linear programming (MILP) is NP-hard in general, pra...
research
∙
10/14/2022
Communication-Efficient Topologies for Decentralized Learning with O(1) Consensus Rate
Decentralized optimization is an emerging paradigm in distributed learni...
research
∙
09/25/2022
On Representing Linear Programs by Graph Neural Networks
Learning to optimize is a rapidly growing area that aims to solve optimi...
research
∙
06/08/2022
Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compression
Recent advances in distributed optimization and learning have shown that...
research
∙
11/08/2021
BlueFog: Make Decentralized Algorithms Practical for Optimization and Deep Learning
Decentralized algorithm is a form of computation that achieves a global ...
research
∙
10/29/2021
Hyperparameter Tuning is All You Need for LISTA
Learned Iterative Shrinkage-Thresholding Algorithm (LISTA) introduces th...
research
∙
10/26/2021
Exponential Graph is Provably Efficient for Decentralized Deep Training
Decentralized SGD is an emerging training method for deep learning known...
research
∙
10/11/2021
Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier Detection
Robust principal component analysis (RPCA) is a critical tool in modern ...
research
∙
06/25/2021
Tighter Analysis of Alternating Stochastic Gradient Method for Stochastic Nested Problems
Stochastic nested optimization, including stochastic compositional, min-...
research
∙
06/02/2021
Learn to Predict Equilibria via Fixed Point Networks
Systems of interacting agents can often be modeled as contextual games, ...
research
∙
05/19/2021
Accelerating Gossip SGD with Periodic Global Averaging
Communication overhead hinders the scalability of large-scale distribute...
research
∙
04/29/2021
Feasibility-based Fixed Point Networks
Inverse problems consist of recovering a signal from a collection of noi...
research
∙
04/25/2021
On the Comparison between Cyclic Sampling and Random Reshuffling
When applying a stochastic/incremental algorithm, one must choose the or...
research
∙
04/24/2021
DecentLaM: Decentralized Momentum SGD for Large-batch Deep Training
The scale of deep learning nowadays calls for efficient distributed trai...
research
∙
03/23/2021
Learning to Optimize: A Primer and A Benchmark
Learning to optimize (L2O) is an emerging approach that leverages machin...
research
∙
03/23/2021
Fixed Point Networks: Implicit Depth Models with Jacobian-Free Backprop
A growing trend in deep learning replaces fixed depth models by approxim...
research
∙
03/22/2021
Provably Correct Optimization and Exploration with Non-linear Policies
Policy optimization methods remain a powerful workhorse in empirical Rei...
research
∙
02/21/2021
A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization
We consider the zeroth-order optimization problem in the huge-scale sett...
research
∙
02/09/2021
A Single-Timescale Stochastic Bilevel Optimization Method
Stochastic bilevel optimization generalizes the classic stochastic optim...
research
∙
12/31/2020
CADA: Communication-Adaptive Distributed Adam
Stochastic gradient descent (SGD) has taken the stage as the primary wor...
research
∙
12/22/2020
Hybrid Federated Learning: Algorithms and Implementation
Federated learning (FL) is a recently proposed distributed machine learn...
research
∙
10/06/2020
SCOBO: Sparsity-Aware Comparison Oracle Based Optimization
We study derivative-free optimization for convex functions where we furt...
research
∙
08/25/2020
Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimization
Stochastic compositional optimization generalizes classic (non-compositi...
research
∙
08/05/2020
Projecting to Manifolds via Unsupervised Learning
We present a new framework, called adversarial projections, for solving ...
research
∙
07/12/2020
VAFL: a Method of Vertical Asynchronous Federated Learning
Horizontal Federated learning (FL) handles multi-client data that share ...
research
∙
05/22/2020
FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data
Federated Learning (FL) has become a popular paradigm for learning from ...
research
∙
03/29/2020
Zeroth-Order Regularized Optimization (ZORO): Approximately Sparse Gradients and Adaptive Sampling
We consider the problem of minimizing a high-dimensional objective funct...
research
∙
03/15/2020
Provably Efficient Exploration for RL with Unsupervised Learning
We study how to use unsupervised learning for efficient exploration in r...
research
∙
03/04/2020
Safeguarded Learned Convex Optimization
Many applications require repeatedly solving a certain type of optimizat...
research
∙
02/26/2020
LASG: Lazily Aggregated Stochastic Gradients for Communication-Efficient Distributed Learning
This paper targets solving distributed machine learning problems such as...
research
∙
12/28/2019
Scaled Relative Graph of Normal Matrices
The Scaled Relative Graph (SRG) by Ryu, Hannah, and Yin (arXiv:1902.0978...
research
∙
12/06/2019
Does Knowledge Transfer Always Help to Learn a Better Policy?
One of the key approaches to save samples when learning a policy for a r...
research
∙
10/24/2019
XPipe: Efficient Pipeline Model Parallelism for Multi-GPU DNN Training
We propose XPipe, an efficient asynchronous pipeline model parallelism a...
research
∙
05/26/2019
ODE Analysis of Stochastic Gradient Methods with Optimism and Anchoring for Minimax Problems and GANs
Despite remarkable empirical success, the training dynamics of generativ...
research
∙
05/14/2019
Plug-and-Play Methods Provably Converge with Properly Trained Denoisers
Plug-and-play (PnP) is a non-convex framework that integrates modern den...
research
∙
12/03/2018
AsyncQVI: Asynchronous-Parallel Q-Value Iteration for Reinforcement Learning with Near-Optimal Sample Complexity
In this paper, we propose AsyncQVI: Asynchronous-Parallel Q-value Iterat...
research
∙
11/22/2018
Markov Chain Block Coordinate Descent
The method of block coordinate gradient descent (BCD) has been a powerfu...
research
∙
09/29/2018
Multilevel Optimal Transport: a Fast Approximation of Wasserstein-1 distances
We propose a fast algorithm for the calculation of the Wasserstein-1 dis...
research
∙
09/12/2018
On Markov Chain Gradient Descent
Stochastic gradient methods are the workhorse (algorithms) of large-scal...
research
∙
08/29/2018
Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds
In recent years, unfolding iterative algorithms as neural networks has b...
research
∙
05/25/2018
LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learning
This paper presents a new class of gradient methods for distributed mach...
research
∙
04/18/2018
Walkman: A Communication-Efficient Random-Walk Algorithm for Decentralized Optimization
This paper addresses consensus optimization problems in a multi-agent ne...
research
∙
04/18/2018