Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement Learning Approach

by   Jingren Xu, et al.

This paper investigates a master unmanned aerial vehicle (MUAV)-powered Internet of Things (IoT) network, in which we propose using a rechargeable auxiliary UAV (AUAV) equipped with an intelligent reflecting surface (IRS) to enhance the communication signals from the MUAV and also leverage the MUAV as a recharging power source. Under the proposed model, we investigate the optimal collaboration strategy of these energy-limited UAVs to maximize the accumulated throughput of the IoT network. Depending on whether there is charging between the two UAVs, two optimization problems are formulated. To solve them, two multi-agent deep reinforcement learning (DRL) approaches are proposed, which are centralized training multi-agent deep deterministic policy gradient (CT-MADDPG) and multi-agent deep deterministic policy option critic (MADDPOC). It is shown that the CT-MADDPG can greatly reduce the requirement on the computing capability of the UAV hardware, and the proposed MADDPOC is able to support low-level multi-agent cooperative learning in the continuous action domains, which has great advantages over the existing option-based hierarchical DRL that only support single-agent learning and discrete actions.



There are no comments yet.


page 1

page 10

page 11


Multi-Agent Deep Reinforcement Learning Based Trajectory Planning for Multi-UAV Assisted Mobile Edge Computing

An unmanned aerial vehicle (UAV)-aided mobile edge computing (MEC) frame...

SREC: Proactive Self-Remedy of Energy-Constrained UAV-Based Networks via Deep Reinforcement Learning

Energy-aware control for multiple unmanned aerial vehicles (UAVs) is one...

Multi-Agent Asynchronous Cooperation with Hierarchical Reinforcement Learning

Hierarchical multi-agent reinforcement learning (MARL) has shown a signi...

Multi-agent Bayesian Deep Reinforcement Learning for Microgrid Energy Management under Communication Failures

Microgrids (MGs) are important players for the future transactive energy...

Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey

Future Internet involves several emerging technologies such as 5G and be...

Proficiency Aware Multi-Agent Actor-Critic for Mixed Aerial and Ground Robot Teaming

Mixed Cooperation and competition are the actual scenarios of deploying ...

Revisiting the Master-Slave Architecture in Multi-Agent Deep Reinforcement Learning

Many tasks in artificial intelligence require the collaboration of multi...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.