Log In Sign Up

Robust Hierarchical Planning with Policy Delegation

by   Tin Lai, et al.

We propose a novel framework and algorithm for hierarchical planning based on the principle of delegation. This framework, the Markov Intent Process, features a collection of skills which are each specialised to perform a single task well. Skills are aware of their intended effects and are able to analyse planning goals to delegate planning to the best-suited skill. This principle dynamically creates a hierarchy of plans, in which each skill plans for sub-goals for which it is specialised. The proposed planning method features on-demand execution—skill policies are only evaluated when needed. Plans are only generated at the highest level, then expanded and optimised when the latest state information is available. The high-level plan retains the initial planning intent and previously computed skills, effectively reducing the computation needed to adapt to environmental changes. We show this planning approach is experimentally very competitive to classic planning and reinforcement learning techniques on a variety of domains, both in terms of solution length and planning time.


page 1

page 2

page 3

page 4


Learning to Plan Hierarchically from Curriculum

We present a framework for learning to plan hierarchically in domains wi...

Discover Life Skills for Planning with Bandits via Observing and Learning How the World Works

We propose a novel approach for planning agents to compose abstract skil...

Sub-policy Adaptation for Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning is a promising approach to long-hori...

Reset-Free Lifelong Learning with Skill-Space Planning

The objective of lifelong reinforcement learning (RL) is to optimize age...

Towards Plug'n Play Task-Level Autonomy for Robotics Using POMDPs and Generative Models

To enable robots to achieve high level objectives, engineers typically w...

Exploring Hierarchy-Aware Inverse Reinforcement Learning

We introduce a new generative model for human planning under the Bayesia...

Learning to Imagine Manipulation Goals for Robot Task Planning

Prospection is an important part of how humans come up with new task pla...