Log In Sign Up

When Deep Reinforcement Learning Meets Federated Learning: Intelligent Multi-Timescale Resource Management for Multi-access Edge Computing in 5G Ultra Dense Network

by   Shuai Yu, et al.

Ultra-dense edge computing (UDEC) has great potential, especially in the 5G era, but it still faces challenges in its current solutions, such as the lack of: i) efficient utilization of multiple 5G resources (e.g., computation, communication, storage and service resources); ii) low overhead offloading decision making and resource allocation strategies; and iii) privacy and security protection schemes. Thus, we first propose an intelligent ultra-dense edge computing (I-UDEC) framework, which integrates blockchain and Artificial Intelligence (AI) into 5G ultra-dense edge computing networks. First, we show the architecture of the framework. Then, in order to achieve real-time and low overhead computation offloading decisions and resource allocation strategies, we design a novel two-timescale deep reinforcement learning (2Ts-DRL) approach, consisting of a fast-timescale and a slow-timescale learning process, respectively. The primary objective is to minimize the total offloading delay and network resource usage by jointly optimizing computation offloading, resource allocation and service caching placement. We also leverage federated learning (FL) to train the 2Ts-DRL model in a distributed manner, aiming to protect the edge devices' data privacy. Simulation results corroborate the effectiveness of both the 2Ts-DRL and FL in the I-UDEC framework and prove that our proposed algorithm can reduce task execution time up to 31.87


Computation Offloading and Resource Allocation in F-RANs: A Federated Deep Reinforcement Learning Approach

The fog radio access network (F-RAN) is a promising technology in which ...

Federated Learning Meets Blockchain in Edge Computing: Opportunities and Challenges

Mobile edge computing (MEC) has been envisioned as a promising paradigm ...

A Resource Allocation Scheme for Energy Demand Management in 6G-enabled Smart Grid

Smart grid (SG) systems enhance grid resilience and efficient operation,...

Caching and Computation Offloading in High Altitude Platform Station (HAPS) Assisted Intelligent Transportation Systems

Edge intelligence, which is a new paradigm to accelerate artificial inte...

Beam Management in Ultra-dense mmWave Network via Federated Reinforcement Learning: An Intelligent and Secure Approach

Deploying ultra-dense networks that operate on millimeter wave (mmWave) ...

Federated Double Deep Q-learning for Joint Delay and Energy Minimization in IoT networks

In this paper, we propose a federated deep reinforcement learning framew...

Deep Reinforcement Learning Based Mode Selection and Resource Management for Green Fog Radio Access Networks

Fog radio access networks (F-RANs) are seen as potential architectures t...