Realtime State Estimation with Tactile and Visual Sensing for Inserting a Suction-held Object

by   Kuan-Ting Yu, et al.

We develop a real-time state estimation system to recover the pose and contact formation of an object relative to its environment. In this paper, we focus on the application of inserting an object picked by a suction cup into a tight space, an enabling technology for robotic packaging. We propose a framework that fuses force and visual sensing for improved accuracy and robustness. Visual sensing is versatile and non-intrusive, but suffers from occlusions and limited accuracy, especially for tasks involving contact. Tactile sensing is local, but provides accuracy and robustness to occlusions. The proposed algorithm to fuse them is based on iSAM, an on-line optimization technique, which we use to incorporate kinematic measurements from the robot, contact geometry of the object and the container, and visual tracking. In this paper, we generalize previous results in planar settings to a 3D task with more complex contact interactions. A key challenge in using force sensing is that we do not observe contact point locations directly. We propose a data-driven method to infer the contact formation, which is then used in real-time by the state estimator. We demonstrate and evaluate the algorithm in a setup instrumented to provide groundtruth.


page 1

page 5

page 7


Extrinsic Contact Sensing with Relative-Motion Tracking from Distributed Tactile Measurements

This paper addresses the localization of contacts of an unknown grasped ...

Joint Inference of Kinematic and Force Trajectories with Visuo-Tactile Sensing

To perform complex tasks, robots must be able to interact with and manip...

A Learning-Based Estimation and Control Framework for Contact-Intensive Tight-Tolerance Tasks

We propose a novel data-driven estimation and control framework for cont...

Fast and Accurate Data-Driven Simulation Framework for Contact-Intensive Tight-Tolerance Robotic Assembly Tasks

We propose a novel fast and accurate simulation framework for contact-in...

MultiSCOPE: Disambiguating In-Hand Object Poses with Proprioception and Tactile Feedback

In this paper, we propose a method for estimating in-hand object poses u...

Legged Robot State-Estimation Through Combined Forward Kinematic and Preintegrated Contact Factors

State-of-the-art robotic perception systems have achieved sufficiently g...

Real-Time Marker Localization Learning for GelStereo Tactile Sensing

Visuotactile sensing technology is becoming more popular in tactile sens...

Please sign up or login with your details

Forgot password? Click here to reset