DeepAI AI Chat
Log In Sign Up

A Monotone Approximate Dynamic Programming Approach for the Stochastic Scheduling, Allocation, and Inventory Replenishment Problem: Applications to Drone and Electric Vehicle B

by   Amin Asadi, et al.

There is a growing interest in using electric vehicles (EVs) and drones for many applications. However, battery-oriented issues, including range anxiety and battery degradation, impede adoption. Battery swap stations are one alternative to reduce these concerns that allow the swap of depleted for full batteries in minutes. We consider the problem of deriving actions at a battery swap station when explicitly considering the uncertain arrival of swap demand, battery degradation, and replacement. We model the operations at a battery swap station using a finite horizon Markov Decision Process model for the stochastic scheduling, allocation, and inventory replenishment problem (SAIRP), which determines when and how many batteries are charged, discharged, and replaced over time. We present theoretical proofs for the monotonicity of the value function and monotone structure of an optimal policy for special SAIRP cases. Due to the curses of dimensionality, we develop a new monotone approximate dynamic programming (ADP) method, which intelligently initializes a value function approximation using regression. In computational tests, we demonstrate the superior performance of the new regression-based monotone ADP method as compared to exact methods and other monotone ADP methods. Further, with the tests, we deduce policy insights for drone swap stations.


Counterexamples on the monotonicity of delay optimal strategies for energy harvesting transmitters

We consider cross-layer design of delay optimal transmission strategies ...

Optimal Cycling of a Heterogenous Battery Bank via Reinforcement Learning

We consider the problem of optimal charging/discharging of a bank of het...

Electric Vehicle Charging Station Search in Stochastic Environments

Electric vehicles are a central component of future mobility systems as ...

Learning to Operate an Electric Vehicle Charging Station Considering Vehicle-grid Integration

The rapid adoption of electric vehicles (EVs) calls for the widespread i...

On the Influence of Charging Stations Spatial Distribution on Aerial Wireless Networks

Using drones for cellular coverage enhancement is a recent technology th...

Exploiting First-Order Regression in Inductive Policy Selection

We consider the problem of computing optimal generalised policies for re...