Ubiquitous Distributed Deep Reinforcement Learning at the Edge: Analyzing Byzantine Agents in Discrete Action Spaces

08/18/2020
by   Wenshuai Zhao, et al.
0

The integration of edge computing in next-generation mobile networks is bringing low-latency and high-bandwidth ubiquitous connectivity to a myriad of cyber-physical systems. This will further boost the increasing intelligence that is being embedded at the edge in various types of autonomous systems, where collaborative machine learning has the potential to play a significant role. This paper discusses some of the challenges in multi-agent distributed deep reinforcement learning that can occur in the presence of byzantine or malfunctioning agents. As the simulation-to-reality gap gets bridged, the probability of malfunctions or errors must be taken into account. We show how wrong discrete actions can significantly affect the collaborative learning effort. In particular, we analyze the effect of having a fraction of agents that might perform the wrong action with a given probability. We study the ability of the system to converge towards a common working policy through the collaborative learning process based on the number of experiences from each of the agents to be aggregated for each policy update, together with the fraction of wrong actions from agents experiencing malfunctions. Our experiments are carried out in a simulation environment using the Atari testbed for the discrete action spaces, and advantage actor-critic (A2C) for the distributed multi-agent training.

READ FULL TEXT

page 1

page 4

page 5

research
08/18/2020

Towards Closing the Sim-to-Real Gap in Collaborative Multi-Robot Deep Reinforcement Learning

Current research directions in deep reinforcement learning include bridg...
research
06/10/2022

Deep Multi-Agent Reinforcement Learning with Hybrid Action Spaces based on Maximum Entropy

Multi-agent deep reinforcement learning has been applied to address a va...
research
02/10/2022

DDA3C: Cooperative Distributed Deep Reinforcement Learning in A Group-Agent System

It can largely benefit the reinforcement learning process of each agent ...
research
11/18/2022

Credit-cognisant reinforcement learning for multi-agent cooperation

Traditional multi-agent reinforcement learning (MARL) algorithms, such a...
research
07/06/2021

Effects of Smart Traffic Signal Control on Air Quality

Adaptive traffic signal control (ATSC) in urban traffic networks poses a...
research
09/02/2022

Semi-Centralised Multi-Agent Reinforcement Learning with Policy-Embedded Training

Centralised training (CT) is the basis for many popular multi-agent rein...
research
09/25/2022

Cooperative Sensing and Heterogeneous Information Fusion in VCPS: A Multi-agent Deep Reinforcement Learning Approach

Cooperative sensing and heterogeneous information fusion are critical to...

Please sign up or login with your details

Forgot password? Click here to reset