Robust Keypoint Detection and Pose Estimation of Robot Manipulators with Self-Occlusions via Sim-to-Real Transfer

10/15/2020
by   Jingpei Lu, et al.
0

Keypoint detection is an essential building block for many robotic applications like motion capture and pose estimation. Historically, keypoints are detected using uniquely engineered markers such as checkerboards, fiducials, or markers. More recently, deep learning methods have been explored as they have the ability to detect user-defined keypoints in a marker-less manner. However, deep neural network (DNN) detectors can have an uneven performance for different manually selected keypoints along the kinematic chain. An example of this can be found on symmetric robotic tools where DNN detectors cannot solve the correspondence problem correctly. In this work, we propose a new and autonomous way to define the keypoint locations that overcomes these challenges. The approach involves finding the optimal set of keypoints on robotic manipulators for robust visual detection. Using a robotic simulator as a medium, our algorithm utilizes synthetic data for DNN training, and the proposed algorithm is used to optimize the selection of keypoints through an iterative approach. The results show that when using the optimized keypoints, the detection performance of the DNNs improved so significantly that they can even be detected in cases of self-occlusion. We further use the optimized keypoints for real robotic applications by using domain randomization to bridge the reality gap between the simulator and the physical world. The physical world experiments show how the proposed method can be applied to the wide-breadth of robotic applications that require visual feedback, such as camera-to-robot calibration, robotic tool tracking, and whole-arm pose estimation.

READ FULL TEXT

page 1

page 3

page 6

page 7

page 8

research
09/27/2018

Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects

Using synthetic data for training deep neural networks for robotic manip...
research
05/02/2021

MarkerPose: Robust Real-time Planar Target Tracking for Accurate Stereo Pose Estimation

Despite the attention marker-less pose estimation has attracted in recen...
research
02/28/2023

Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfer

Solving the camera-to-robot pose is a fundamental requirement for vision...
research
03/02/2022

ParaPose: Parameter and Domain Randomization Optimization for Pose Estimation using Synthetic Data

Pose estimation is the task of determining the 6D position of an object ...
research
08/15/2023

Learning Better Keypoints for Multi-Object 6DoF Pose Estimation

We investigate the impact of pre-defined keypoints for pose estimation, ...
research
04/14/2022

Sim-to-Real 6D Object Pose Estimation via Iterative Self-training for Robotic Bin-picking

In this paper, we propose an iterative self-training framework for sim-t...
research
06/09/2020

SEKD: Self-Evolving Keypoint Detection and Description

Researchers have attempted utilizing deep neural network (DNN) to learn ...

Please sign up or login with your details

Forgot password? Click here to reset