Fast Learning for Renewal Optimization in Online Task Scheduling

07/18/2020
by   Michael J. Neely, et al.
0

This paper considers online optimization of a renewal-reward system. A controller performs a sequence of tasks back-to-back. Each task has a random vector of parameters, called the task type vector, that affects the task processing options and also affects the resulting reward and time duration of the task. The probability distribution for the task type vector is unknown and the controller must learn to make efficient decisions so that time average reward converges to optimality. Prior work on such renewal optimization problems leaves open the question of optimal convergence time. This paper develops an algorithm with an optimality gap that decays like O(1/√(k)), where k is the number of tasks processed. The same algorithm is shown to have faster O(log(k)/k) performance when the system satisfies a strong concavity property. The proposed algorithm uses an auxiliary variable that is updated according to a classic Robbins-Monro iteration. It makes online scheduling decisions at the start of each renewal frame based on this variable and on the observed task type. A matching converse is obtained for the strongly concave case by constructing an example system for which all algorithms have performance at best Ω(log(k)/k). A matching Ω(1/√(k)) converse is also shown for the general case without strong concavity.

READ FULL TEXT
research
02/27/2022

Age Optimal Sampling Under Unknown Delay Statistics

This paper revisits the problem of sampling and transmitting status upda...
research
02/08/2018

Adaptive online scheduling of tasks with anytime property on heterogeneous resources

An acceptable response time of a server is an important aspect in many c...
research
06/09/2021

A Lyapunov-Based Methodology for Constrained Optimization with Bandit Feedback

In a wide variety of applications including online advertising, contract...
research
08/02/2022

Digital Twin-Assisted Efficient Reinforcement Learning for Edge Task Scheduling

Task scheduling is a critical problem when one user offloads multiple di...
research
01/10/2018

Learning Aided Optimization for Energy Harvesting Devices with Outdated State Information

This paper considers utility optimal power control for energy harvesting...
research
08/30/2020

A Meta-Learning Control Algorithm with Provable Finite-Time Guarantees

In this work we provide provable regret guarantees for an online meta-le...

Please sign up or login with your details

Forgot password? Click here to reset