Optimizing the Long-Term Average Reward for Continuing MDPs: A Technical Report

04/13/2021
by   Chao Xu, et al.
0

Recently, we have struck the balance between the information freshness, in terms of age of information (AoI), experienced by users and energy consumed by sensors, by appropriately activating sensors to update their current status in caching enabled Internet of Things (IoT) networks [1]. To solve this problem, we cast the corresponding status update procedure as a continuing Markov Decision Process (MDP) (i.e., without termination states), where the number of state-action pairs increases exponentially with respect to the number of considered sensors and users. Moreover, to circumvent the curse of dimensionality, we have established a methodology for designing deep reinforcement learning (DRL) algorithms to maximize (resp. minimize) the average reward (resp. cost), by integrating R-learning, a tabular reinforcement learning (RL) algorithm tailored for maximizing the long-term average reward, and traditional DRL algorithms, initially developed to optimize the discounted long-term cumulative reward rather than the average one. In this technical report, we would present detailed discussions on the technical contributions of this methodology.

READ FULL TEXT

page 1

page 2

page 3

page 5

page 6

page 7

page 8

page 9

research
06/13/2021

Optimal Status Update for Caching Enabled IoT Networks: A Dueling Deep R-Network Approach

In the Internet of Things (IoT) networks, caching is a promising techniq...
research
03/01/2020

AoI and Energy Consumption Oriented Dynamic Status Updating in Caching Enabled IoT Networks

Caching has been regarded as a promising technique to alleviate energy c...
research
10/23/2020

Deep Reinforcement Learning for IoT Networks: Age of Information and Energy Cost Tradeoff

In most Internet of Things (IoT) networks, edge nodes are commonly used ...
research
04/16/2020

A Deep Reinforcement Learning Approach for Dynamic Contents Caching in HetNets

The recent development in Internet of Things necessitates caching of dyn...
research
12/31/2020

Autonomous Maintenance in IoT Networks via AoI-driven Deep Reinforcement Learning

Internet of Things (IoT) with its growing number of deployed devices and...
research
02/19/2021

A Reinforcement Learning Approach to Age of Information in Multi-User Networks with HARQ

Scheduling the transmission of time-sensitive information from a source ...
research
08/06/2019

Age of Information-Aware Radio Resource Management in Vehicular Networks: A Proactive Deep Reinforcement Learning Perspective

In this paper, we investigate the problem of age of information (AoI)-aw...

Please sign up or login with your details

Forgot password? Click here to reset