KeyPose: Multi-view 3D Labeling and Keypoint Estimation for Transparent Objects

12/05/2019
by   Xingyu Liu, et al.
5

Estimating the 3D pose of desktop objects is crucial for applications such as robotic manipulation. Finding the depth of the object is an important part of this task, both for training and prediction, and is usually accomplished with a depth sensor or markers in a motion-capture system. For transparent or highly reflective objects, such methods are not feasible without impinging on the resultant image of the object. Hence, many existing methods restrict themselves to opaque, lambertian objects that give good returns from RGBD sensors. In this paper we address two problems: first, establish an easy method for capturing and labeling 3D keypoints on desktop objects with a stereo sensor (no special depth sensor required); and second, develop a deep method, called KeyPose, that learns to accurately predict 3D keypoints on objects, including challenging ones such as transparent objects. To showcase the performance of the method, we create and employ a dataset of 15 clear objects in 5 classes, with 48k 3D-keypoint labeled images. We train both instance and category models, and show generalization to new textures, poses, and objects. KeyPose surpasses state-of-the-art performance in 3D pose estimation on this dataset, sometimes by a wide margin, and even in cases where the competing method is provided with registered depth. We will release a public version of the data capture and labeling pipeline, the transparent object database, and the KeyPose training and evaluation code.

READ FULL TEXT

page 4

page 8

page 10

page 11

page 12

page 13

page 14

page 15

research
02/22/2023

MVTrans: Multi-View Perception of Transparent Objects

Transparent object perception is a crucial skill for applications such a...
research
11/03/2022

StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS

Most existing methods for category-level pose estimation rely on object ...
research
06/03/2016

On Recognizing Transparent Objects in Domestic Environments Using Fusion of Multiple Sensor Modalities

Current object recognition methods fail on object sets that include both...
research
11/15/2022

Grasping the Inconspicuous

Transparent objects are common in day-to-day life and hence find many ap...
research
07/11/2022

A4T: Hierarchical Affordance Detection for Transparent Objects Depth Reconstruction and Manipulation

Transparent objects are widely used in our daily lives and therefore rob...
research
04/09/2019

3D Object Instance Recognition and Pose Estimation Using Triplet Loss with Dynamic Margin

In this paper, we address the problem of 3D object instance recognition ...
research
11/29/2022

Compliant Suction Gripper with Seamless Deployment and Retraction for Robust Picking against Depth and Tilt Errors

Applying suction grippers in unstructured environments is a challenging ...

Please sign up or login with your details

Forgot password? Click here to reset