Lyapunov-guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks

by   Suzhi Bi, et al.

Opportunistic computation offloading is an effective method to improve the computation performance of mobile-edge computing (MEC) networks under dynamic edge environment. In this paper, we consider a multi-user MEC network with time-varying wireless channels and stochastic user task data arrivals in sequential time frames. In particular, we aim to design an online computation offloading algorithm to maximize the network data processing capability subject to the long-term data queue stability and average power constraints. The online algorithm is practical in the sense that the decisions for each time frame are made without the assumption of knowing future channel conditions and data arrivals. We formulate the problem as a multi-stage stochastic mixed integer non-linear programming (MINLP) problem that jointly determines the binary offloading (each user computes the task either locally or at the edge server) and system resource allocation decisions in sequential time frames. To address the coupling in the decisions of different time frames, we propose a novel framework, named LyDROO, that combines the advantages of Lyapunov optimization and deep reinforcement learning (DRL). Specifically, LyDROO first applies Lyapunov optimization to decouple the multi-stage stochastic MINLP into deterministic per-frame MINLP subproblems. By doing so, it guarantees to satisfy all the long-term constraints by solving the per-frame subproblems that are much smaller in size. Then, LyDROO integrates model-based optimization and model-free DRL to solve the per-frame MINLP problems with low computational complexity. Simulation results show that under various network setups, the proposed LyDROO achieves optimal computation performance while stabilizing all queues in the system. Besides, it induces very low execution latency that is particularly suitable for real-time implementation in fast fading environments.



There are no comments yet.


page 9

page 12

page 16

page 20

page 21

page 22

page 25

page 29


Stable Online Computation Offloading via Lyapunov-guided Deep Reinforcement Learning

In this paper, we consider a multi-user mobile-edge computing (MEC) netw...

Deep Reinforcement Learning for Online Offloading in Wireless Powered Mobile-Edge Computing Networks

In this paper, we consider a wireless powered mobile-edge computing (MEC...

An Integrated Optimization-Learning Framework for Online Combinatorial Computation Offloading in MEC Networks

Mobile edge computing (MEC) is a promising paradigm to accommodate the i...

Deep Reinforcement Learning for Backscatter-Aided Data Offloading in Mobile Edge Computing

Wireless network optimization has been becoming very challenging as the ...

Energy-Efficient Online Data Sensing and Processing in Wireless Powered Edge Computing Systems

This paper focuses on developing energy-efficient online data processing...

Online Cognitive Data Sensing and Processing Optimization in Energy-harvesting Edge Computing Systems

Mobile edge computing (MEC) has recently become a prevailing technique t...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.