Long-Horizon Manipulation of Unknown Objects via Task and Motion Planning with Estimated Affordances

by   Aidan Curtis, et al.

We present a strategy for designing and building very general robot manipulation systems involving the integration of a general-purpose task-and-motion planner with engineered and learned perception modules that estimate properties and affordances of unknown objects. Such systems are closed-loop policies that map from RGB images, depth images, and robot joint encoder measurements to robot joint position commands. We show that following this strategy a task-and-motion planner can be used to plan intelligent behaviors even in the absence of a priori knowledge regarding the set of manipulable objects, their geometries, and their affordances. We explore several different ways of implementing such perceptual modules for segmentation, property detection, shape estimation, and grasp generation. We show how these modules are integrated within the PDDLStream task and motion planning framework. Finally, we demonstrate that this strategy can enable a single system to perform a wide variety of real-world multi-step manipulation tasks, generalizing over a broad class of objects, object arrangements, and goals, without any prior knowledge of the environment and without re-training.


page 1

page 7

page 8

page 9

page 10

page 15

page 16

page 17


Task Allocation for Multi-Robot Task and Motion Planning: a case for Object Picking in Cluttered Workspaces

We present an AND/OR graph-based, integrated multi-robot task and motion...

Combined Task and Motion Planning for a Dual-arm Robot to Use a Suction Cup Tool

This paper proposes a combined task and motion planner for a dual-arm ro...

ADAMANT: A Pipeline for Adaptable Manipulation Tasks

This paper presents ADAMANT, a set of software modules that provides gra...

Robotic Perception-motion Synergy for Novel Rope Wrapping Tasks

This paper introduces a novel and general method to address the problem ...

Learning compositional models of robot skills for task and motion planning

The objective of this work is to augment the basic abilities of a robot ...

Predicting Stable Configurations for Semantic Placement of Novel Objects

Human environments contain numerous objects configured in a variety of a...

kPAM-SC: Generalizable Manipulation Planning using KeyPoint Affordance and Shape Completion

Manipulation planning is the task of computing robot trajectories that m...

Please sign up or login with your details

Forgot password? Click here to reset