Optimal Network Control in Partially-Controllable Networks

01/06/2019
by   Qingkai Liang, et al.
0

The effectiveness of many optimal network control algorithms (e.g., BackPressure) relies on the premise that all of the nodes are fully controllable. However, these algorithms may yield poor performance in a partially-controllable network where a subset of nodes are uncontrollable and use some unknown policy. Such a partially-controllable model is of increasing importance in real-world networked systems such as overlay-underlay networks. In this paper, we design optimal network control algorithms that can stabilize a partially-controllable network. We first study the scenario where uncontrollable nodes use a queue-agnostic policy, and propose a low-complexity throughput-optimal algorithm, called Tracking-MaxWeight (TMW), which enhances the original MaxWeight algorithm with an explicit learning of the policy used by uncontrollable nodes. Next, we investigate the scenario where uncontrollable nodes use a queue-dependent policy and the problem is formulated as an MDP with unknown queueing dynamics. We propose a new reinforcement learning algorithm, called Truncated Upper Confidence Reinforcement Learning (TUCRL), and prove that TUCRL achieves tunable three-way tradeoffs between throughput, delay and convergence rate.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/14/2020

RL-QN: A Reinforcement Learning Framework for Optimal Control of Queueing Systems

With the rapid advance of information technology, network systems have b...
research
05/28/2021

A New Algorithm for the LQR Problem with Partially Unknown Dynamics

We consider an LQR optimal control problem with partially unknown dynami...
research
10/27/2021

Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks

This paper presents a problem in power networks that creates an exciting...
research
10/12/2017

Designing Low-Complexity Heavy-Traffic Delay-Optimal Load Balancing Schemes: Theory to Algorithms

We establish a unified analytical framework for load balancing systems, ...
research
02/10/2018

Laplacian Dynamics on Cographs: Controllability Analysis through Joins and Unions

In this paper, we examine the controllability of Laplacian dynamic netwo...
research
03/27/2023

Model-Based Reinforcement Learning with Isolated Imaginations

World models learn the consequences of actions in vision-based interacti...
research
08/30/2021

Trustworthy AI for Process Automation on a Chylla-Haase Polymerization Reactor

In this paper, genetic programming reinforcement learning (GPRL) is util...

Please sign up or login with your details

Forgot password? Click here to reset