DynoPlan: Combining Motion Planning and Deep Neural Network based Controllers for Safe HRL

06/24/2019
by   Daniel Angelov, et al.
0

Many realistic robotics tasks are best solved compositionally, through control architectures that sequentially invoke primitives and achieve error correction through the use of loops and conditionals taking the system back to alternative earlier states. Recent end-to-end approaches to task learning attempt to directly learn a single controller that solves an entire task, but this has been difficult for complex control tasks that would have otherwise required a diversity of local primitive moves, and the resulting solutions are also not easy to inspect for plan monitoring purposes. In this work, we aim to bridge the gap between hand designed and learned controllers, by representing each as an option in a hybrid hierarchical Reinforcement Learning framework - DynoPlan. We extend the options framework by adding a dynamics model and the use of a nearness-to-goal heuristic, derived from demonstrations. This translates the optimization of a hierarchical policy controller to a problem of planning with a model predictive controller. By unrolling the dynamics of each option and assessing the expected value of each future state, we can create a simple switching controller for choosing the optimal policy within a constrained time horizon similarly to hill climbing heuristic search. The individual dynamics model allows each option to iterate and be activated independently of the specific underlying instantiation, thus allowing for a mix of motion planning and deep neural network based primitives. We can assess the safety regions of the resulting hybrid controller by investigating the initiation sets of the different options, and also by reasoning about the completeness and performance guarantees of the underpinning motion planners.

READ FULL TEXT
research
07/18/2019

Composing Diverse Policies for Temporally Extended Tasks

Temporally extended and sequenced robot motion tasks are often character...
research
09/05/2020

Learning Topological Motion Primitives for Knot Planning

In this paper, we approach the challenging problem of motion planning fo...
research
08/22/2017

Reinforcement Learning in POMDPs with Memoryless Options and Option-Observation Initiation Sets

Many real-world reinforcement learning problems have a hierarchical natu...
research
03/19/2020

Goal-Conditioned End-to-End Visuomotor Control for Versatile Skill Primitives

Visuomotor control (VMC) is an effective means of achieving basic manipu...
research
05/15/2020

Enhancing Lattice-based Motion Planning with Introspective Learning and Reasoning

Lattice-based motion planning is a hybrid planning method where a plan m...
research
07/09/2019

Hybrid system identification using switching density networks

Behaviour cloning is a commonly used strategy for imitation learning and...
research
02/27/2019

From explanation to synthesis: Compositional program induction for learning from demonstration

Hybrid systems are a compact and natural mechanism with which to address...

Please sign up or login with your details

Forgot password? Click here to reset