DeepAI AI Chat
Log In Sign Up

Reinforcement Learning for Datacenter Congestion Control

02/18/2021
by   Chen Tessler, et al.
0

We approach the task of network congestion control in datacenters using Reinforcement Learning (RL). Successful congestion control algorithms can dramatically improve latency and overall network throughput. Until today, no such learning-based algorithms have shown practical potential in this domain. Evidently, the most popular recent deployments rely on rule-based heuristics that are tested on a predetermined set of benchmarks. Consequently, these heuristics do not generalize well to newly-seen scenarios. Contrarily, we devise an RL-based algorithm with the aim of generalizing to different configurations of real-world datacenter networks. We overcome challenges such as partial-observability, non-stationarity, and multi-objectiveness. We further propose a policy gradient algorithm that leverages the analytical structure of the reward function to approximate its derivative and improve stability. We show that this scheme outperforms alternative popular RL approaches, and generalizes to scenarios that were not seen during training. Our experiments, conducted on a realistic simulator that emulates communication networks' behavior, exhibit improved performance concurrently on the multiple considered metrics compared to the popular algorithms deployed today in real datacenters. Our algorithm is being productized to replace heuristics in some of the largest datacenters in the world.

READ FULL TEXT

page 1

page 2

page 3

page 4

11/22/2022

A Reinforcement Learning Approach to Optimize Available Network Bandwidth Utilization

Efficient data transfers over high-speed, long-distance shared networks ...
07/05/2022

Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs

Cloud datacenters are exponentially growing both in numbers and size. Th...
12/24/2018

Iroko: A Framework to Prototype Reinforcement Learning for Data Center Traffic Control

Recent networking research has identified that data-driven congestion co...
10/09/2019

MVFST-RL: An Asynchronous RL Framework for Congestion Control with Delayed Actions

Effective network congestion control strategies are key to keeping the I...
05/26/2022

RACE: A Reinforcement Learning Framework for Improved Adaptive Control of NoC Channel Buffers

Network-on-chip (NoC) architectures rely on buffers to store flits to co...
10/22/2020

When Machine Learning Meets Congestion Control: A Survey and Comparison

Machine learning (ML) has seen a significant surge and uptake across man...
04/26/2019

Reinforcement Learning Based Orchestration for Elastic Services

Due to the highly variable execution context in which edge services run,...