DeepAI AI Chat
Log In Sign Up

Spatially and Seamlessly Hierarchical Reinforcement Learning for State Space and Policy space in Autonomous Driving

by   Jaehyun Kim, et al.

Despite advances in hierarchical reinforcement learning, its applications to path planning in autonomous driving on highways are challenging. One reason is that conventional hierarchical reinforcement learning approaches are not amenable to autonomous driving due to its riskiness: the agent must move avoiding multiple obstacles such as other agents that are highly unpredictable, thus safe regions are small, scattered, and changeable over time. To overcome this challenge, we propose a spatially hierarchical reinforcement learning method for state space and policy space. The high-level policy selects not only behavioral sub-policy but also regions to pay mind to in state space and for outline in policy space. Subsequently, the low-level policy elaborates the short-term goal position of the agent within the outline of the region selected by the high-level command. The network structure and optimization suggested in our method are as concise as those of single-level methods. Experiments on the environment with various shapes of roads showed that our method finds the nearly optimal policies from early episodes, outperforming a baseline hierarchical reinforcement learning method, especially in narrow and complex roads. The resulting trajectories on the roads were similar to those of human strategies on the behavioral planning level.


page 7

page 11

page 13

page 14


Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving

Tactical decision making for autonomous driving is challenging due to th...

SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments

Designing a safe and human-like decision-making system for an autonomous...

Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learning

Goal-conditioned hierarchical reinforcement learning (HRL) is a promisin...

Trajectory Planning for Autonomous Vehicles Using Hierarchical Reinforcement Learning

Planning safe trajectories under uncertain and dynamic conditions makes ...

Behaviorally Diverse Traffic Simulation via Reinforcement Learning

Traffic simulators are important tools in autonomous driving development...

Hierarchical Reinforcement Learning with Abductive Planning

One of the key challenges in applying reinforcement learning to real-lif...