Communication-Efficient Federated Bilevel Optimization with Local and Global Lower Level Problems

02/13/2023
by   Junyi Li, et al.
0

Bilevel Optimization has witnessed notable progress recently with new emerging efficient algorithms, yet it is underexplored in the Federated Learning setting. It is unclear how the challenges of Federated Learning affect the convergence of bilevel algorithms. In this work, we study Federated Bilevel Optimization problems. We first propose the FedBiO algorithm that solves the hyper-gradient estimation problem efficiently, then we propose FedBiOAcc to accelerate FedBiO. FedBiO has communication complexity O(ϵ^-1.5) with linear speed up, while FedBiOAcc achieves communication complexity O(ϵ^-1), sample complexity O(ϵ^-1.5) and also the linear speed up. We also study Federated Bilevel Optimization problems with local lower level problems, and prove that FedBiO and FedBiOAcc converges at the same rate with some modification.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/03/2022

Local Stochastic Bilevel Optimization with Momentum-Based Variance Reduction

Bilevel Optimization has witnessed notable progress recently with new em...
research
02/09/2023

Communication-Efficient Federated Hypergradient Computation via Aggregated Iterative Differentiation

Federated bilevel optimization has attracted increasing attention due to...
research
02/02/2022

Communication Efficient Federated Learning for Generalized Linear Bandits

Contextual bandit algorithms have been recently studied under the federa...
research
08/08/2023

Federated Zeroth-Order Optimization using Trajectory-Informed Surrogate Gradients

Federated optimization, an emerging paradigm which finds wide real-world...
research
05/18/2023

The Blessing of Heterogeneity in Federated Q-learning: Linear Speedup and Beyond

When the data used for reinforcement learning (RL) are collected by mult...
research
06/08/2023

Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates

Distributed and federated learning algorithms and techniques associated ...
research
03/17/2021

Sample-based Federated Learning via Mini-batch SSCA

In this paper, we investigate unconstrained and constrained sample-based...

Please sign up or login with your details

Forgot password? Click here to reset