AI Chat AI Image Generator AI Video Text to Speech

On the Convergence of Momentum-Based Algorithms for Federated Stochastic Bilevel Optimization Problems

04/28/2022

∙

by Hongchang Gao, et al.

∙

∙

In this paper, we studied the federated stochastic bilevel optimization problem. In particular, we developed two momentum-based algorithms for optimizing this kind of problem. In addition, we established the convergence rate of these two algorithms, providing their sample and communication complexities. To the best of our knowledge, this is the first work achieving such favorable theoretical results.

page 1

page 2

page 3

page 4

research

∙ 02/10/2023

Achieving Linear Speedup in Non-IID Federated Bilevel Learning

Federated bilevel optimization has received increasing attention in vari...

0 Minhui Huang, et al. ∙

research

∙ 04/20/2023

Federated Compositional Deep AUC Maximization

Federated learning has attracted increasing attention due to the promise...

0 Xinwen Zhang, et al. ∙

research

∙ 03/17/2021

Sample-based Federated Learning via Mini-batch SSCA

In this paper, we investigate unconstrained and constrained sample-based...

0 Chencheng Ye, et al. ∙

research

∙ 06/08/2021

Provably Faster Algorithms for Bilevel Optimization

Bilevel optimization has been widely applied in many important machine l...

0 Junjie Yang, et al. ∙

research

∙ 06/19/2021

STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning

Federated Learning (FL) refers to the paradigm where multiple worker nod...

5 Prashant Khanduri, et al. ∙

research

∙ 07/30/2020

Momentum Q-learning with Finite-Sample Convergence Guarantee

Existing studies indicate that momentum ideas in conventional optimizati...

8 Bowen Weng, et al. ∙

research

∙ 08/06/2023

Serverless Federated AUPRC Optimization for Multi-Party Collaborative Imbalanced Data Mining

Multi-party collaborative training, such as distributed learning and fed...

0 Xidong Wu, et al. ∙