DeepAI AI Chat
Log In Sign Up

Distributed Uplink Beamforming in Cell-Free Networks Using Deep Reinforcement Learning

06/26/2020
by   Firas Fredj, et al.
0

The emergence of new wireless technologies together with the requirement of massive connectivity results in several technical issues such as excessive interference, high computational demand for signal processing, and lengthy processing delays. In this work, we propose several beamforming techniques for an uplink cell-free network with centralized, semi-distributed, and fully distributed processing, all based on deep reinforcement learning (DRL). First, we propose a fully centralized beamforming method that uses the deep deterministic policy gradient algorithm (DDPG) with continuous space. We then enhance this method by enabling distributed experience at access points (AP). Indeed, we develop a beamforming scheme that uses the distributed distributional deterministic policy gradients algorithm (D4PG) with the APs representing the distributed agents. Finally, to decrease the computational complexity, we propose a fully distributed beamforming scheme that divides the beamforming computations among APs. The results show that the D4PG scheme with distributed experience achieves the best performance irrespective of the network size. Furthermore, the proposed distributed beamforming technique performs better than the DDPG algorithm with centralized learning only for small-scale networks. The performance superiority of the DDPG model becomes more evident as the number of APs and/or users increases. Moreover, during the operation stage, all DRL models demonstrate a significantly shorter processing time than that of the conventional gradient descent (GD) solution.

READ FULL TEXT

page 20

page 22

01/29/2020

Multiple Access in Dynamic Cell-Free Networks: Outage Performance and Deep Reinforcement Learning-Based Design

In future cell-free (or cell-less) wireless networks, a large number of ...
02/05/2021

Deep Reinforcement Learning for Energy-Efficient Beamforming Design in Cell-Free Networks

Cell-free network is considered as a promising architecture for satisfyi...
03/17/2021

Self-Organizing mmWave MIMO Cell-Free Networks With Hybrid Beamforming: A Hierarchical DRL-Based Design

In a cell-free wireless network, distributed access points (APs) jointly...
05/25/2020

Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications

Intelligent reflecting surface (IRS) is a promising technology to assist...
06/30/2021

Limited-Fronthaul Cell-Free Hybrid Beamforming with Distributed Deep Neural Network

Cell-free massive MIMO (CF-mMIMO) systems represent a promising approach...
04/21/2020

STDPG: A Spatio-Temporal Deterministic Policy Gradient Agent for Dynamic Routing in SDN

Dynamic routing in software-defined networking (SDN) can be viewed as a ...
04/23/2018

Distributed Distributional Deterministic Policy Gradients

This work adopts the very successful distributional perspective on reinf...