DeepAI AI Chat
Log In Sign Up

Communication-Efficient Federated Bilevel Optimization with Local and Global Lower Level Problems

by   Junyi Li, et al.

Bilevel Optimization has witnessed notable progress recently with new emerging efficient algorithms, yet it is underexplored in the Federated Learning setting. It is unclear how the challenges of Federated Learning affect the convergence of bilevel algorithms. In this work, we study Federated Bilevel Optimization problems. We first propose the FedBiO algorithm that solves the hyper-gradient estimation problem efficiently, then we propose FedBiOAcc to accelerate FedBiO. FedBiO has communication complexity O(ϵ^-1.5) with linear speed up, while FedBiOAcc achieves communication complexity O(ϵ^-1), sample complexity O(ϵ^-1.5) and also the linear speed up. We also study Federated Bilevel Optimization problems with local lower level problems, and prove that FedBiO and FedBiOAcc converges at the same rate with some modification.


page 1

page 2

page 3

page 4


Local Stochastic Bilevel Optimization with Momentum-Based Variance Reduction

Bilevel Optimization has witnessed notable progress recently with new em...

Communication-Efficient Federated Hypergradient Computation via Aggregated Iterative Differentiation

Federated bilevel optimization has attracted increasing attention due to...

Communication Efficient Federated Learning for Generalized Linear Bandits

Contextual bandit algorithms have been recently studied under the federa...

The Blessing of Heterogeneity in Federated Q-learning: Linear Speedup and Beyond

When the data used for reinforcement learning (RL) are collected by mult...

Sample-based Federated Learning via Mini-batch SSCA

In this paper, we investigate unconstrained and constrained sample-based...

Lower Bounds and Optimal Algorithms for Personalized Federated Learning

In this work, we consider the optimization formulation of personalized f...

Federated Learning with Randomized Douglas-Rachford Splitting Methods

In this paper, we develop two new algorithms, called, FedDR and asyncFed...