Indirect Object-to-Robot Pose Estimation from an External Monocular RGB Camera

by   Jonathan Tremblay, et al.

We present a robotic grasping system that uses a single external monocular RGB camera as input. The object-to-robot pose is computed indirectly by combining the output of two neural networks: one that estimates the object-to-camera pose, and another that estimates the robot-to-camera pose. Both networks are trained entirely on synthetic data, relying on domain randomization to bridge the sim-to-real gap. Because the latter network performs online camera calibration, the camera can be moved freely during execution without affecting the quality of the grasp. Experimental results analyze the effect of camera placement, image resolution, and pose refinement in the context of grasping several household objects. We also present results on a new set of 28 textured household toy grocery objects, which have been selected to be accessible to other researchers. To aid reproducibility of the research, we offer 3D scanned textured models, along with pre-trained weights for pose estimation.



There are no comments yet.


page 1

page 3

page 4

page 7


Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects

Using synthetic data for training deep neural networks for robotic manip...

Pose Estimation of Specular and Symmetrical Objects

In the robotic industry, specular and textureless metallic components ar...

Camera-to-Robot Pose Estimation from a Single Image

We present an approach for estimating the pose of a camera with respect ...

SilhoNet: An RGB Method for 3D Object Pose Estimation and Grasp Planning

Autonomous robot manipulation often involves both estimating the pose of...

Single-view robot pose and joint angle estimation via render compare

We introduce RoboPose, a method to estimate the joint angles and the 6D ...

Vision-Only Robot Navigation in a Neural Radiance World

Neural Radiance Fields (NeRFs) have recently emerged as a powerful parad...

LyRN (Lyapunov Reaching Network): A Real-Time Closed Loop approach from Monocular Vision

We propose a closed-loop, multi-instance control algorithm for visually ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.