Composing Diverse Policies for Temporally Extended Tasks

by   Daniel Angelov, et al.

Temporally extended and sequenced robot motion tasks are often characterized by discontinuous switches between different types of local dynamics. These change-points can be exploited to build approximate models of the interleaving regions, which in turn allow the design of region-specific controllers. These can then be combined to create the initiation state-space of a final policy. However, such a pipeline can become challenging to implement for combinatorially complex, temporarily extended tasks - especially so when sub-controllers work on different information streams, time scales and action spaces. In this paper, we introduce a method that can compose diverse policies based on scripted motion planning, dynamic motion primitives and neural networks. In order to do this, we extend the options framework to introduce a per-option dynamics module and a global function that evaluates a goal metric. Additionally, we can leverage expert demonstrations to sequence these local policies, converting the learning problem in hierarchical reinforcement learning to a planning problem at inference time. We first illustrate the core concepts with an MDP benchmark, and then with a physical gear assembly task solved on a PR2 robot. We show that the proposed approach successfully discovers the optimal sequence of policies and solves both tasks efficiently.


page 1

page 7

page 8


DynoPlan: Combining Motion Planning and Deep Neural Network based Controllers for Safe HRL

Many realistic robotics tasks are best solved compositionally, through c...

Leveraging Task Knowledge for Robot Motion Planning Under Uncertainty

Noisy observations coupled with nonlinear dynamics pose one of the bigge...

Efficient Hierarchical Robot Motion Planning Under Uncertainty and Hybrid Dynamics

Noisy observations coupled with nonlinear dynamics pose one of the bigge...

Continuous Motion Planning with Temporal Logic Specifications using Deep Neural Networks

In this paper, we propose a model-free reinforcement learning method to ...

Improved Soft Duplicate Detection in Search-Based Motion Planning

Search-based techniques have shown great success in motion planning prob...

Temporally Extended Successor Representations

We present a temporally extended variation of the successor representati...

"Guess what I'm doing": Extending legibility to sequential decision tasks

In this paper we investigate the notion of legibility in sequential deci...

Please sign up or login with your details

Forgot password? Click here to reset