Multi-Agent Deep Reinforcement Learning for Cost- and Delay-Sensitive Virtual Network Function Placement and Routing

by   Shaoyang Wang, et al.

This paper proposes an effective and novel multiagent deep reinforcement learning (MADRL)-based method for solving the joint virtual network function (VNF) placement and routing (P R), where multiple service requests with differentiated demands are delivered at the same time. The differentiated demands of the service requests are reflected by their delay- and cost-sensitive factors. We first construct a VNF P R problem to jointly minimize a weighted sum of service delay and resource consumption cost, which is NP-complete. Then, the joint VNF P R problem is decoupled into two iterative subtasks: placement subtask and routing subtask. Each subtask consists of multiple concurrent parallel sequential decision processes. By invoking the deep deterministic policy gradient method and multi-agent technique, an MADRL-P R framework is designed to perform the two subtasks. The new joint reward and internal rewards mechanism is proposed to match the goals and constraints of the placement and routing subtasks. We also propose the parameter migration-based model-retraining method to deal with changing network topologies. Corroborated by experiments, the proposed MADRL-P R framework is superior to its alternatives in terms of service cost and delay, and offers higher flexibility for personalized service demands. The parameter migration-based model-retraining method can efficiently accelerate convergence under moderate network topology changes.


page 4

page 5

page 6

page 7

page 10

page 11

page 14

page 17


On the Cost-Optimality Trade-off for Service Function Chain Reconfiguration

Optimal placement of Virtual Network Functions (VNFs) in virtualized dat...

MSDF: A Deep Reinforcement Learning Framework for Service Function Chain Migration

Under dynamic traffic, service function chain (SFC) migration is conside...

Joint Resource Management for MC-NOMA: A Deep Reinforcement Learning Approach

This paper presents a novel and effective deep reinforcement learning (D...

Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning

In delay-sensitive industrial internet of things (IIoT) applications, th...

Multi-Timescale Online Optimization of Network Function Virtualization for Service Chaining

Network Function Virtualization (NFV) can cost-efficiently provide netwo...

Mean-Field Game and Reinforcement Learning MEC Resource Provisioning for SFC

In this paper, we address the resource provisioning problem for service ...

Depth-Optimized Delay-Aware Tree (DO-DAT) for Virtual Network Function Placement

With the constant increase in demand for data connectivity, network serv...

Please sign up or login with your details

Forgot password? Click here to reset