DeepAI AI Chat
Log In Sign Up

Multi-Task Option Learning and Discovery for Stochastic Path Planning

by   Naman Shah, et al.

This paper addresses the problem of reliably and efficiently solving broad classes of long-horizon stochastic path planning problems. Starting with a vanilla RL formulation with a stochastic dynamics simulator and an occupancy matrix of the environment, our approach computes useful options with policies as well as high-level paths that compose the discovered options. Our main contributions are (1) data-driven methods for creating abstract states that serve as endpoints for helpful options, (2) methods for computing option policies using auto-generated option guides in the form of dense pseudo-reward functions, and (3) an overarching algorithm for composing the computed options. We show that this approach yields strong guarantees of executability and solvability: under fairly general conditions, the computed option guides lead to composable option policies and consequently ensure downward refinability. Empirical evaluation on a range of robots, environments, and tasks shows that this approach effectively transfers knowledge across related tasks and that it outperforms existing approaches by a significant margin.


page 7

page 8

page 9


Reward-Respecting Subtasks for Model-Based Reinforcement Learning

To achieve the ambitious goals of artificial intelligence, reinforcement...

Discovery of Options via Meta-Learned Subgoals

Temporal abstractions in the form of options have been shown to help rei...

Optimal Options for Multi-Task Reinforcement Learning Under Time Constraints

Reinforcement learning can greatly benefit from the use of options as a ...

Reinforcement Learning in POMDPs with Memoryless Options and Option-Observation Initiation Sets

Many real-world reinforcement learning problems have a hierarchical natu...

SOAC: The Soft Option Actor-Critic Architecture

The option framework has shown great promise by automatically extracting...

PODNet: A Neural Network for Discovery of Plannable Options

Learning from demonstration has been widely studied in machine learning ...

Toward Discovering Options that Achieve Faster Planning

We propose a new objective for option discovery that emphasizes the comp...