Building 3D Object Models during Manipulation by Reconstruction-Aware Trajectory Optimization

by   Kanrun Huang, et al.

Object shape provides important information for robotic manipulation; for instance, selecting an effective grasp depends on both the global and local shape of the object of interest, while reaching into clutter requires accurate surface geometry to avoid unintended contact with the environment. Model-based 3D object manipulation is a widely studied problem; however, obtaining the accurate 3D object models for multiple objects often requires tedious work. In this letter, we exploit Gaussian process implicit surfaces (GPIS) extracted from RGB-D sensor data to grasp an unknown object. We propose a reconstruction-aware trajectory optimization that makes use of the extracted GPIS model plan a motion to improve the ability to estimate the object's 3D geometry, while performing a pick-and-place action. We present a probabilistic approach for a robot to autonomously learn and track the object, while achieve the manipulation task. We use a sampling-based trajectory generation method to explore the unseen parts of the object using the estimated conditional entropy of the GPIS model. We validate our method with physical robot experiments across eleven different objects of varying shape from the YCB object dataset. Our experiments show that our reconstruction-aware trajectory optimization provides higher-quality 3D object reconstruction when compared with directly solving the manipulation task or using a heuristic to view unseen portions of the object.


page 1

page 2

page 3

page 4

page 5

page 6

page 7

page 8


Neural Field Representations of Articulated Objects for Robotic Manipulation Planning

Traditional approaches for manipulation planning rely on an explicit geo...

Relaxed-Rigidity Constraints: Kinematic Trajectory Optimization and Collision Avoidance for In-Grasp Manipulation

This paper proposes a novel approach to performing in-grasp manipulation...

Efficient Visuo-Haptic Object Shape Completion for Robot Manipulation

For robot manipulation, a complete and accurate object shape is desirabl...

A Convex-Combinatorial Model for Planar Caging

Caging is a promising tool which allows a robot to manipulate an object ...

Active Tapping via Gaussian Process for Efficient Unknown Object Surface Reconstruction

Object surface reconstruction brings essential benefits to robot graspin...

Annotation Scaffolds for Object Modeling and Manipulation

We present and evaluate an approach for human-in-the-loop specification ...

Prepare the Chair for the Bear! Robot Imagination of Sitting Affordance to Reorient Previously Unseen Chairs

In this letter, a paradigm for the classification and manipulation of pr...

Please sign up or login with your details

Forgot password? Click here to reset