Deep Reinforcement Learning for Real-Time Optimization in NB-IoT Networks

12/21/2018
by   Nan Jiang, et al.
0

NarrowBand-Internet of Things (NB-IoT) is an emerging cellular-based technology that offers a range of flexible configurations for massive IoT radio access from groups of devices with heterogeneous requirements. A configuration specifies the amount of radio resource allocated to each group of devices for random access and for data transmission. Assuming no knowledge of the traffic statistics, there exists an important challenge in "how to determine the configuration that maximizes the long-term average number of served IoT devices at each Transmission Time Interval (TTI) in an online fashion". Given the complexity of searching for optimal configuration, we first develop real-time configuration selection based on the tabular Q-learning (tabular-Q), the Linear Approximation based Q-learning (LA-Q), and the Deep Neural Network based Q-learning (DQN) in the single-parameter single-group scenario. Our results show that the proposed reinforcement learning based approaches considerably outperform the conventional heuristic approaches based on load estimation (LE-URC) in terms of the number of served IoT devices. This result also indicates that LA-Q and DQN can be good alternatives for tabular-Q to achieve almost the same performance with much less training time. We further advance LA-Q and DQN via Actions Aggregation (AA-LA-Q and AA-DQN) and via Cooperative Multi-Agent learning (CMA-DQN) for the multi-parameter multi-group scenario, thereby solve the problem that Q-learning agents do not converge in high-dimensional configurations. In this scenario, the superiority of the proposed Q-learning approaches over the conventional LE-URC approach significantly improves with the increase of configuration dimensions, and the CMA-DQN approach outperforms the other approaches in both throughput and training efficiency.

READ FULL TEXT
research
10/27/2018

Cooperative Deep Reinforcement Learning for Multiple-Group NB-IoT Networks Optimization

NarrowBand-Internet of Things (NB-IoT) is an emerging cellular-based tec...
research
10/27/2018

Cooperative Deep Reinforcement Learning for Multiple Groups NB-IoT Networks Optimization

NarrowBand-Internet of Things (NB-IoT) is an emerging cellular-based tec...
research
06/19/2019

Multi-user Resource Control with Deep Reinforcement Learning in IoT Edge Computing

By leveraging the concept of mobile edge computing (MEC), massive amount...
research
05/10/2019

Intelligent User Association for Symbiotic Radio Networks using Deep Reinforcement Learning

In this paper, we are interested in symbiotic radio networks, in which a...
research
04/30/2018

A Non-parametric Multi-stage Learning Framework for Cognitive Spectrum Access in IoT Networks

Given the increasing number of devices that is going to get connected to...
research
12/10/2021

SmartCon: Deep Probabilistic Learning Based Intelligent Link-Configuration in Narrowband-IoT Towards 5G and B5G

To enhance the coverage and transmission reliability, repetitions adopte...
research
06/19/2021

Algorithm Unrolling for Massive Access via Deep Neural Network with Theoretical Guarantee

Massive access is a critical design challenge of Internet of Things (IoT...

Please sign up or login with your details

Forgot password? Click here to reset