Integrating Task-Motion Planning with Reinforcement Learning for Robust Decision Making in Mobile Robots

11/21/2018
by   Yuqian Jiang, et al.
0

Task-motion planning (TMP) addresses the problem of efficiently generating executable and low-cost task plans in a discrete space such that the (initially unknown) action costs are determined by motion plans in a corresponding continuous space. However, a task-motion plan can be sensitive to unexpected domain uncertainty and changes, leading to suboptimal behaviors or execution failures. In this paper, we propose a novel framework, TMP-RL, which is an integration of TMP and reinforcement learning (RL) from the execution experience, to solve the problem of robust task-motion planning in dynamic and uncertain domains. TMP-RL features two nested planning-learning loops. In the inner TMP loop, the robot generates a low-cost, feasible task-motion plan by iteratively planning in the discrete space and updating relevant action costs evaluated by the motion planner in continuous space. In the outer loop, the plan is executed, and the robot learns from the execution experience via model-free RL, to further improve its task-motion plans. RL in the outer loop is more accurate to the current domain but also more expensive, and using less costly task and motion planning leads to a jump-start for learning in the real world. Our approach is evaluated on a mobile service robot conducting navigation tasks in an office area. Results show that TMP-RL approach significantly improves adaptability and robustness (in comparison to TMP methods) and leads to rapid convergence (in comparison to task planning (TP)-RL methods). We also show that TMP-RL can reuse learned values to smoothly adapt to new scenarios during long-term deployments.

READ FULL TEXT

page 6

page 8

research
06/01/2019

Harnessing Reinforcement Learning for Neural Motion Planning

Motion planning is an essential component in most of today's robotic app...
research
08/31/2021

A review of mobile robot motion planning methods: from classical motion planning workflows to reinforcement learning-based architectures

Motion planning is critical to realize the autonomous operation of mobil...
research
09/21/2020

CMAX++ : Leveraging Experience in Planning and Execution using Inaccurate Models

Given access to accurate dynamical models, modern planning approaches ar...
research
04/10/2021

MPTP: Motion-Planning-aware Task Planning for Navigation in Belief Space

We present an integrated Task-Motion Planning (TMP) framework for naviga...
research
02/05/2021

An advantage actor-critic algorithm for robotic motion planning in dense and dynamic scenarios

Intelligent robots provide a new insight into efficiency improvement in ...
research
02/22/2022

Visually Grounded Task and Motion Planning for Mobile Manipulation

Task and motion planning (TAMP) algorithms aim to help robots achieve ta...
research
08/28/2021

Anytime Stochastic Task and Motion Policies

In order to solve complex, long-horizon tasks, intelligent robots need t...

Please sign up or login with your details

Forgot password? Click here to reset