Deep Reinforcement Learning with Discrete Normalized Advantage Functions for Resource Management in Network Slicing

06/10/2019
by   Chen Qi, et al.
0

Network slicing promises to provision diversified services with distinct requirements in one infrastructure. Deep reinforcement learning (e.g., deep Q-learning, DQL) is assumed to be an appropriate algorithm to solve the demand-aware inter-slice resource management issue in network slicing by regarding the varying demands and the allocated bandwidth as the environment state and the action, respectively. However, allocating bandwidth in a finer resolution usually implies larger action space, and unfortunately DQL fails to quickly converge in this case. In this paper, we introduce discrete normalized advantage functions (DNAF) into DQL, by separating the Q-value function as a state-value function term and an advantage term and exploiting a deterministic policy gradient descent (DPGD) algorithm to avoid the unnecessary calculation of Q-value for every state-action pair. Furthermore, as DPGD only works in continuous action space, we embed a k-nearest neighbor algorithm into DQL to quickly find a valid action in the discrete space nearest to the DPGD output. Finally, we verify the faster convergence of the DNAF-based DQL through extensive simulations.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

research
05/10/2019

GAN-based Deep Distributional Reinforcement Learning for Resource Management in Network Slicing

Network slicing is a key technology in 5G communications system, which a...
research
05/29/2022

Representation Gap in Deep Reinforcement Learning

Deep reinforcement learning gives the promise that an agent learns good ...
research
08/11/2021

Graph Attention Network-based Multi-agent Reinforcement Learning for Slicing Resource Management in Dense Cellular Network

Network slicing (NS) management devotes to providing various services to...
research
06/18/2020

Reducing Estimation Bias via Weighted Delayed Deep Deterministic Policy Gradient

The overestimation phenomenon caused by function approximation is a well...
research
10/10/2018

Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space

Most existing deep reinforcement learning (DRL) frameworks consider eith...
research
12/24/2015

Deep Reinforcement Learning in Large Discrete Action Spaces

Being able to reason in an environment with a large number of discrete a...

Please sign up or login with your details

Forgot password? Click here to reset