Log In Sign Up

Discover Life Skills for Planning with Bandits via Observing and Learning How the World Works

by   Tin Lai, et al.

We propose a novel approach for planning agents to compose abstract skills via observing and learning from historical interactions with the world. Our framework operates in a Markov state-space model via a set of actions under unknown pre-conditions. We formulate skills as high-level abstract policies that propose action plans based on the current state. Each policy learns new plans by observing the states' transitions while the agent interacts with the world. Such an approach automatically learns new plans to achieve specific intended effects, but the success of such plans is often dependent on the states in which they are applicable. Therefore, we formulate the evaluation of such plans as infinitely many multi-armed bandit problems, where we balance the allocation of resources on evaluating the success probability of existing arms and exploring new options. The result is a planner capable of automatically learning robust high-level skills under a noisy environment; such skills implicitly learn the action pre-condition without explicit knowledge. We show that this planning approach is experimentally very competitive in high-dimensional state space domains.


Robust Hierarchical Planning with Policy Delegation

We propose a novel framework and algorithm for hierarchical planning bas...

Learning to Plan Hierarchically from Curriculum

We present a framework for learning to plan hierarchically in domains wi...

Learning Features and Abstract Actions for Computing Generalized Plans

Generalized planning is concerned with the computation of plans that sol...

MDP Abstraction with Successor Features

Abstraction plays an important role for generalisation of knowledge and ...

Sim-to-Real Task Planning and Execution from Perception via Reactivity and Recovery

Zero-shot execution of unseen robotic tasks is an important problem in r...

Option Discovery in Hierarchical Reinforcement Learning using Spatio-Temporal Clustering

This paper introduces an automated skill acquisition framework in reinfo...

Planning with Large Language Models via Corrective Re-prompting

Extracting the common sense knowledge present in Large Language Models (...