Energy Minimization in UAV-Aided Networks: Actor-Critic Learning for Constrained Scheduling Optimization

by   Yaxiong Yuan, et al.

In unmanned aerial vehicle (UAV) applications, the UAV's limited energy supply and storage have triggered the development of intelligent energy-conserving scheduling solutions. In this paper, we investigate energy minimization for UAV-aided communication networks by jointly optimizing data-transmission scheduling and UAV hovering time. The formulated problem is combinatorial and non-convex with bilinear constraints. To tackle the problem, firstly, we provide an optimal relax-and-approximate solution and develop a near-optimal algorithm. Both the proposed solutions are served as offline performance benchmarks but might not be suitable for online operation. To this end, we develop a solution from a deep reinforcement learning (DRL) aspect. The conventional RL/DRL, e.g., deep Q-learning, however, is limited in dealing with two main issues in constrained combinatorial optimization, i.e., exponentially increasing action space and infeasible actions. The novelty of solution development lies in handling these two issues. To address the former, we propose an actor-critic-based deep stochastic online scheduling (AC-DSOS) algorithm and develop a set of approaches to confine the action space. For the latter, we design a tailored reward function to guarantee the solution feasibility. Numerical results show that, by consuming equal magnitude of time, AC-DSOS is able to provide feasible solutions and saves 29.94 with a conventional deep actor-critic method. Compared to the developed near-optimal algorithm, AC-DSOS consumes around 10 the computational time from minute-level to millisecond-level.


An Actor-Critic-Based UAV-BSs Deployment Method for Dynamic Environments

In this paper, the real-time deployment of unmanned aerial vehicles (UAV...

UAV Trajectory Planning in Wireless Sensor Networks for Energy Consumption Minimization by Deep Reinforcement Learning

Unmanned aerial vehicles (UAVs) have emerged as a promising candidate so...

SREC: Proactive Self-Remedy of Energy-Constrained UAV-Based Networks via Deep Reinforcement Learning

Energy-aware control for multiple unmanned aerial vehicles (UAVs) is one...

Combinatorial Keyword Recommendations for Sponsored Search with Deep Reinforcement Learning

In sponsored search, keyword recommendations help advertisers to achieve...

A Deep Reinforcement Learning Approach for Constrained Online Logistics Route Assignment

As online shopping prevails and e-commerce platforms emerge, there is a ...

Learning to Generate All Feasible Actions

Several machine learning (ML) applications are characterized by searchin...

A Hierarchical Deep Actor-Critic Learning Method for Joint Distribution System State Estimation

Due to increasing penetration of volatile distributed photovoltaic (PV) ...

Please sign up or login with your details

Forgot password? Click here to reset