Communication-Efficient Consensus Mechanism for Federated Reinforcement Learning

01/30/2022

∙

The paper considers independent reinforcement learning (IRL) for multi-agent decision-making process in the paradigm of federated learning (FL). We show that FL can clearly improve the policy performance of IRL in terms of training efficiency and stability. However, since the policy parameters are trained locally and aggregated iteratively through a central server in FL, frequent information exchange incurs a large amount of communication overheads. To reach a good balance between improving the model's convergence performance and reducing the required communication and computation overheads, this paper proposes a system utility function and develops a consensus-based optimization scheme on top of the periodic averaging method, which introduces the consensus algorithm into FL for the exchange of a model's local gradients. This paper also provides novel convergence guarantees for the developed method, and demonstrates its superior effectiveness and efficiency in improving the system utility value through theoretical analyses and numerical simulation results.

READ FULL TEXT

Communication-Efficient Consensus Mechanism for Federated Reinforcement Learning

The Gradient Convergence Bound of Federated Multi-Agent Reinforcement Learning with Efficient Communication

A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learning

Communication-Efficient Federated Learning with Binary Neural Networks

Performance Optimization for Variable Bitwidth Federated Learning in Wireless Networks

Secure and Efficient Federated Learning Through Layering and Sharding Blockchain

The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining

Decentralized and Model-Free Federated Learning: Consensus-Based Distillation in Function Space

Communication-Efficient Consensus Mechanism for Federated Reinforcement Learning

Related Research

The Gradient Convergence Bound of Federated Multi-Agent Reinforcement Learning with Efficient Communication

A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learning

Communication-Efficient Federated Learning with Binary Neural Networks

Performance Optimization for Variable Bitwidth Federated Learning in Wireless Networks

Secure and Efficient Federated Learning Through Layering and Sharding Blockchain

The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining

Decentralized and Model-Free Federated Learning: Consensus-Based Distillation in Function Space