DeepAI AI Chat
Log In Sign Up

Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications

by   Jiaye Lin, et al.

Intelligent reflecting surface (IRS) is a promising technology to assist downlink information transmissions from a multi-antenna access point (AP) to a receiver. In this paper, we minimize the AP's transmit power by a joint optimization of the AP's active beamforming and the IRS's passive beamforming. Due to uncertain channel conditions, we formulate a robust power minimization problem subject to the receiver's signal-to-noise ratio (SNR) requirement and the IRS's power budget constraint. We propose a deep reinforcement learning (DRL) approach that can adapt the beamforming strategies from past experiences. To improve the learning performance, we derive a convex approximation as a lower bound on the robust problem, which is integrated into the DRL framework and thus promoting a novel optimization-driven deep deterministic policy gradient (DDPG) approach. In particular, when the DDPG algorithm generates a part of the action (e.g., passive beamforming), we can use the model-based convex approximation to optimize the other part (e.g., active beamforming) of the action more efficiently. Our simulation results demonstrate that the optimization-driven DDPG algorithm can improve both the learning rate and reward performance significantly compared to the conventional model-free DDPG algorithm.


Intelligent Reflecting Surface Enhanced Wireless Network via Joint Active and Passive Beamforming

Intelligent reflecting surface (IRS) is envisioned to be a new and revol...

Optimization-driven Machine Learning for Intelligent Reflecting Surfaces Assisted Wireless Networks

Intelligent reflecting surface (IRS) has been recently employed to resha...

Distributed Uplink Beamforming in Cell-Free Networks Using Deep Reinforcement Learning

The emergence of new wireless technologies together with the requirement...

IRS-Assisted Ambient Backscatter Communications Utilizing Deep Reinforcement Learning

We consider an ambient backscatter communication (AmBC) system aided by ...