Experience-driven Networking: A Deep Reinforcement Learning based Approach

01/17/2018
by   Zhiyuan Xu, et al.
0

Modern communication networks have become very complicated and highly dynamic, which makes them hard to model, predict and control. In this paper, we develop a novel experience-driven approach that can learn to well control a communication network from its own experience rather than an accurate mathematical model, just as a human learns a new skill (such as driving, swimming, etc). Specifically, we, for the first time, propose to leverage emerging Deep Reinforcement Learning (DRL) for enabling model-free control in communication networks; and present a novel and highly effective DRL-based control framework, DRL-TE, for a fundamental networking problem: Traffic Engineering (TE). The proposed framework maximizes a widely-used utility function by jointly learning network environment and its dynamics, and making decisions under the guidance of powerful Deep Neural Networks (DNNs). We propose two new techniques, TE-aware exploration and actor-critic-based prioritized experience replay, to optimize the general DRL framework particularly for TE. To validate and evaluate the proposed framework, we implemented it in ns-3, and tested it comprehensively with both representative and randomly generated network topologies. Extensive packet-level simulation results show that 1) compared to several widely-used baseline methods, DRL-TE significantly reduces end-to-end delay and consistently improves the network utility, while offering better or comparable throughput; 2) DRL-TE is robust to network changes; and 3) DRL-TE consistently outperforms a state-ofthe-art DRL method (for continuous control), Deep Deterministic Policy Gradient (DDPG), which, however, does not offer satisfying performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/02/2018

Model-Free Control for Distributed Stream Data Processing using Deep Reinforcement Learning

In this paper, we focus on general-purpose Distributed Stream Data Proce...
research
04/21/2020

STDPG: A Spatio-Temporal Deterministic Policy Gradient Agent for Dynamic Routing in SDN

Dynamic routing in software-defined networking (SDN) can be viewed as a ...
research
06/02/2021

Deep Reinforcement Learning-based UAV Navigation and Control: A Soft Actor-Critic with Hindsight Experience Replay Approach

In this paper, we propose SACHER (soft actor-critic (SAC) with hindsight...
research
06/24/2020

Model-Free Voltage Regulation of Unbalanced Distribution Network Based on Surrogate Model and Deep Reinforcement Learning

Accurate knowledge of the distribution system topology and parameters is...
research
06/16/2022

GMI-DRL: Empowering Multi-GPU Deep Reinforcement Learning with GPU Spatial Multiplexing

With the increasing popularity of robotics in industrial control and aut...
research
09/15/2017

Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning

Cooling system plays a key role in modern data center. Developing an opt...
research
08/09/2022

Automating DBSCAN via Deep Reinforcement Learning

DBSCAN is widely used in many scientific and engineering fields because ...

Please sign up or login with your details

Forgot password? Click here to reset