DROP: Deep relocating option policy for optimal ride-hailing vehicle repositioning

09/09/2021
by   Xinwu Qian, et al.
0

In a ride-hailing system, an optimal relocation of vacant vehicles can significantly reduce fleet idling time and balance the supply-demand distribution, enhancing system efficiency and promoting driver satisfaction and retention. Model-free deep reinforcement learning (DRL) has been shown to dynamically learn the relocating policy by actively interacting with the intrinsic dynamics in large-scale ride-hailing systems. However, the issues of sparse reward signals and unbalanced demand and supply distribution place critical barriers in developing effective DRL models. Conventional exploration strategy (e.g., the ϵ-greedy) may barely work under such an environment because of dithering in low-demand regions distant from high-revenue regions. This study proposes the deep relocating option policy (DROP) that supervises vehicle agents to escape from oversupply areas and effectively relocate to potentially underserved areas. We propose to learn the Laplacian embedding of a time-expanded relocation graph, as an approximation representation of the system relocation policy. The embedding generates task-agnostic signals, which in combination with task-dependent signals, constitute the pseudo-reward function for generating DROPs. We present a hierarchical learning framework that trains a high-level relocation policy and a set of low-level DROPs. The effectiveness of our approach is demonstrated using a custom-built high-fidelity simulator with real-world trip record data. We report that DROP significantly improves baseline models with 15.7 hourly revenue and can effectively resolve the dithering issue in low-demand areas.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/22/2023

AutoVRL: A High Fidelity Autonomous Ground Vehicle Simulator for Sim-to-Real Deep Reinforcement Learning

Deep Reinforcement Learning (DRL) enables cognitive Autonomous Ground Ve...
research
10/05/2020

A Distributed Model-Free Ride-Sharing Approach for Joint Matching, Pricing, and Dispatching using Deep Reinforcement Learning

Significant development of ride-sharing services presents a plethora of ...
research
02/13/2021

Equilibrium Inverse Reinforcement Learning for Ride-hailing Vehicle Network

Ubiquitous mobile computing have enabled ride-hailing services to collec...
research
08/05/2021

Deep Reinforcement Learning for Continuous Docking Control of Autonomous Underwater Vehicles: A Benchmarking Study

Docking control of an autonomous underwater vehicle (AUV) is a task that...
research
05/25/2021

Safe Model-based Off-policy Reinforcement Learning for Eco-Driving in Connected and Automated Hybrid Electric Vehicles

Connected and Automated Hybrid Electric Vehicles have the potential to r...
research
12/12/2022

Decentralized cooperative perception for autonomous vehicles: Learning to value the unknown

Recently, we have been witnesses of accidents involving autonomous vehic...

Please sign up or login with your details

Forgot password? Click here to reset