kPAM-SC: Generalizable Manipulation Planning using KeyPoint Affordance and Shape Completion

09/16/2019
by   Wei Gao, et al.
15

Manipulation planning is the task of computing robot trajectories that move a set of objects to their target configuration while satisfying physically feasibility. In contrast to existing works that assume known object templates, we are interested in manipulation planning for a category of objects with potentially unknown instances and large intra-category shape variation. To achieve it, we need an object representation with which the manipulation planner can reason about both the physical feasibility and desired object configuration, while being generalizable to novel instances. The widely-used pose representation is not suitable, as representing an object with a parameterized transformation from a fixed template cannot capture large intra-category shape variation. Hence, we propose a new hybrid object representation consisting of semantic keypoint and dense geometry (a point cloud or mesh) as the interface between the perception module and motion planner. Leveraging advances in learning-based keypoint detection and shape completion, both dense geometry and keypoints can be perceived from raw sensor input. Using the proposed hybrid object representation, we formulate the manipulation task as a motion planning problem which encodes both the object target configuration and physical feasibility for a category of objects. In this way, many existing manipulation planners can be generalized to categories of objects, and the resulting perception-to-action manipulation pipeline is robust to large intra-category shape variation. Extensive hardware experiments demonstrate our pipeline can produce robot trajectories that accomplish tasks with never-before-seen objects.

READ FULL TEXT

page 1

page 2

page 3

page 5

research
03/15/2019

kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation

We would like robots to achieve purposeful manipulation by placing any i...
research
02/11/2021

kPAM 2.0: Feedback Control for Category-Level Robotic Manipulation

In this paper, we explore generalizable, perception-to-action robotic ma...
research
10/16/2020

Manipulation-Oriented Object Perception in Clutter through Affordance Coordinate Frames

In order to enable robust operation in unstructured environments, robots...
research
08/09/2021

Long-Horizon Manipulation of Unknown Objects via Task and Motion Planning with Estimated Affordances

We present a strategy for designing and building very general robot mani...
research
02/23/2023

Category-level Shape Estimation for Densely Cluttered Objects

Accurately estimating the shape of objects in dense clutters makes impor...
research
07/20/2019

Inferring Occluded Geometry Improves Performance when Retrieving an Object from Dense Clutter

Object search -- the problem of finding a target object in a cluttered s...
research
01/02/2020

Fast Perception, Planning, and Execution for a Robotic Butler: Wheeled Humanoid M-Hubo

As the aging population grows at a rapid rate, there is an ever growing ...

Please sign up or login with your details

Forgot password? Click here to reset