Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning

by   Mohammad Akbari, et al.

In delay-sensitive industrial internet of things (IIoT) applications, the age of information (AoI) is employed to characterize the freshness of information. Meanwhile, the emerging network function virtualization provides flexibility and agility for service providers to deliver a given network service using a sequence of virtual network functions (VNFs). However, suitable VNF placement and scheduling in these schemes is NP-hard and finding a globally optimal solution by traditional approaches is complex. Recently, deep reinforcement learning (DRL) has appeared as a viable way to solve such problems. In this paper, we first utilize single agent low-complex compound action actor-critic RL to cover both discrete and continuous actions and jointly minimize VNF cost and AoI in terms of network resources under end-to end Quality of Service constraints. To surmount the single-agent capacity limitation for learning, we then extend our solution to a multi-agent DRL scheme in which agents collaborate with each other. Simulation results demonstrate that single-agent schemes significantly outperform the greedy algorithm in terms of average network cost and AoI. Moreover, multi-agent solution decreases the average cost by dividing the tasks between the agents. However, it needs more iterations to be learned due to the requirement on the agents collaboration.



page 1

page 4

page 12

page 13

page 14


Federated Multi-Agent Actor-Critic Learning for Age Sensitive Mobile Edge Computing

As an emerging technique, mobile edge computing (MEC) introduces a new p...

Autonomous Maintenance in IoT Networks via AoI-driven Deep Reinforcement Learning

Internet of Things (IoT) with its growing number of deployed devices and...

Age-driven Joint Sampling and Non-slot Based Scheduling for Industrial Internet of Things

Effective control of time-sensitive industrial applications depends on t...

Multi-agent Bayesian Deep Reinforcement Learning for Microgrid Energy Management under Communication Failures

Microgrids (MGs) are important players for the future transactive energy...

Multi-Agent Deep Reinforcement Learning for Cost- and Delay-Sensitive Virtual Network Function Placement and Routing

This paper proposes an effective and novel multiagent deep reinforcement...

Embedding the Minimum Cost SFC with End-to-end Delay Constraint

Many network applications, especially the multimedia applications, often...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.