DHRL-FNMR: An Intelligent Multicast Routing Approach Based on Deep Hierarchical Reinforcement Learning in SDN

by   Miao Ye, et al.

The optimal multicast tree problem in the Software-Defined Networking (SDN) multicast routing is an NP-hard combinatorial optimization problem. Although existing SDN intelligent solution methods, which are based on deep reinforcement learning, can dynamically adapt to complex network link state changes, these methods are plagued by problems such as redundant branches, large action space, and slow agent convergence. In this paper, an SDN intelligent multicast routing algorithm based on deep hierarchical reinforcement learning is proposed to circumvent the aforementioned problems. First, the multicast tree construction problem is decomposed into two sub-problems: the fork node selection problem and the construction of the optimal path from the fork node to the destination node. Second, based on the information characteristics of SDN global network perception, the multicast tree state matrix, link bandwidth matrix, link delay matrix, link packet loss rate matrix, and sub-goal matrix are designed as the state space of intrinsic and meta controllers. Then, in order to mitigate the excessive action space, our approach constructs different action spaces at the upper and lower levels. The meta-controller generates an action space using network nodes to select the fork node, and the intrinsic controller uses the adjacent edges of the current node as its action space, thus implementing four different action selection strategies in the construction of the multicast tree. To facilitate the intelligent agent in constructing the optimal multicast tree with greater speed, we developed alternative reward strategies that distinguish between single-step node actions and multi-step actions towards multiple destination nodes.


page 14

page 16


DRL-M4MR: An Intelligent Multicast Routing Approach Based on DQN Deep Reinforcement Learning in SDN

Traditional multicast routing methods have some problems in constructing...

Intelligent multicast routing method based on multi-agent deep reinforcement learning in SDWN

Multicast communication technology is widely applied in wireless environ...

Reinforcement Learning Based Transmission Range Control (RL-TRC) in SD-WSN with Moving Sensors

Routing in Software-Defined Wireless sensor networks (SD-WSNs) can be ei...

An Intelligent SDWN Routing Algorithm Based on Network Situational Awareness and Deep Reinforcement Learning

Due to the highly dynamic changes in wireless network topologies, effici...

A New Intelligent Cross-Domain Routing Method in SDN Based on a Proposed Multiagent Reinforcement Learning Algorithm

Message transmission and message synchronization for multicontroller int...

Optimizing V2V Unicast Communication Transmission with Reinforcement Learning and Vehicle Clustering

Efficient routing algorithms based on vehicular ad hoc networks (VANETs)...

DeltaPath: dataflow-based high-performance incremental routing

Routing controllers must react quickly to failures, reconfigurations and...

Please sign up or login with your details

Forgot password? Click here to reset