DeepRMSA: A Deep Reinforcement Learning Framework for Routing, Modulation and Spectrum Assignment in Elastic Optical Networks

05/06/2019
by   Xiaoliang Chen, et al.
0

This paper proposes DeepRMSA, a deep reinforcement learning framework for routing, modulation and spectrum assignment (RMSA) in elastic optical networks (EONs). DeepRMSA learns the correct online RMSA policies by parameterizing the policies with deep neural networks (DNNs) that can sense complex EON states. The DNNs are trained with experiences of dynamic lightpath provisioning. We first modify the asynchronous advantage actor-critic algorithm and present an episode-based training mechanism for DeepRMSA, namely, DeepRMSA-EP. DeepRMSA-EP divides the dynamic provisioning process into multiple episodes (each containing the servicing of a fixed number of lightpath requests) and performs training by the end of each episode. The optimization target of DeepRMSA-EP at each step of servicing a request is to maximize the cumulative reward within the rest of the episode. Thus, we obviate the need for estimating the rewards related to unknown future states. To overcome the instability issue in the training of DeepRMSA-EP due to the oscillations of cumulative rewards, we further propose a window-based flexible training mechanism, i.e., DeepRMSA-FLX. DeepRMSA-FLX attempts to smooth out the oscillations by defining the optimization scope at each step as a sliding window, and ensuring that the cumulative rewards always include rewards from a fixed number of requests. Evaluations with the two sample topologies show that DeepRMSA-FLX can effectively stabilize the training while achieving blocking probability reductions of more than 20.3

READ FULL TEXT
research
07/05/2022

Resource Allocation in Multicore Elastic Optical Networks: A Deep Reinforcement Learning Approach

A deep reinforcement learning approach is applied, for the first time, t...
research
05/17/2021

Dynamic Routing and Spectrum Assignment based on the Availability of Consecutive Sub-channels in Flexi-grid Optical Networks

Using Optical Orthogonal Frequency Multiplexing (O-OFDM), variable bandw...
research
02/23/2023

Dynamic realization of miscellaneous profile services in elastic optical networks using spectrum partitioning

Optical backbone networks are required to be highly dynamic in supportin...
research
05/15/2022

RoMFAC: A Robust Mean-Field Actor-Critic Reinforcement Learning against Adversarial Perturbations on States

Deep reinforcement learning methods for multi-agent systems make optimal...
research
12/15/2018

How the Network Topology, Traffic Distribution, and Routing Scheme Impact on the Spectrum Usage in Elastic Optical Networks

Elastic Optical Network (EON) has been considered as a promising optical...
research
05/24/2022

Concurrent Credit Assignment for Data-efficient Reinforcement Learning

The capability to widely sample the state and action spaces is a key ing...
research
09/24/2021

Distributed Deep Reinforcement Learning for Adaptive Medium Access and Modulation in Shared Spectrum

Spectrum scarcity has led to growth in the use of unlicensed spectrum fo...

Please sign up or login with your details

Forgot password? Click here to reset