Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning

05/10/2021
by   Mohammad Akbari, et al.
4

In delay-sensitive industrial internet of things (IIoT) applications, the age of information (AoI) is employed to characterize the freshness of information. Meanwhile, the emerging network function virtualization provides flexibility and agility for service providers to deliver a given network service using a sequence of virtual network functions (VNFs). However, suitable VNF placement and scheduling in these schemes is NP-hard and finding a globally optimal solution by traditional approaches is complex. Recently, deep reinforcement learning (DRL) has appeared as a viable way to solve such problems. In this paper, we first utilize single agent low-complex compound action actor-critic RL to cover both discrete and continuous actions and jointly minimize VNF cost and AoI in terms of network resources under end-to end Quality of Service constraints. To surmount the single-agent capacity limitation for learning, we then extend our solution to a multi-agent DRL scheme in which agents collaborate with each other. Simulation results demonstrate that single-agent schemes significantly outperform the greedy algorithm in terms of average network cost and AoI. Moreover, multi-agent solution decreases the average cost by dividing the tasks between the agents. However, it needs more iterations to be learned due to the requirement on the agents collaboration.

READ FULL TEXT

page 1

page 4

page 12

page 13

page 14

research
12/28/2020

Federated Multi-Agent Actor-Critic Learning for Age Sensitive Mobile Edge Computing

As an emerging technique, mobile edge computing (MEC) introduces a new p...
research
12/31/2020

Autonomous Maintenance in IoT Networks via AoI-driven Deep Reinforcement Learning

Internet of Things (IoT) with its growing number of deployed devices and...
research
05/09/2022

Age-driven Joint Sampling and Non-slot Based Scheduling for Industrial Internet of Things

Effective control of time-sensitive industrial applications depends on t...
research
11/22/2021

Multi-agent Bayesian Deep Reinforcement Learning for Microgrid Energy Management under Communication Failures

Microgrids (MGs) are important players for the future transactive energy...
research
06/24/2022

Multi-Agent Deep Reinforcement Learning for Cost- and Delay-Sensitive Virtual Network Function Placement and Routing

This paper proposes an effective and novel multiagent deep reinforcement...
research
10/09/2020

Embedding the Minimum Cost SFC with End-to-end Delay Constraint

Many network applications, especially the multimedia applications, often...

Please sign up or login with your details

Forgot password? Click here to reset