Log In Sign Up

Towards Learning to Detect and Predict Contact Events on Vision-based Tactile Sensors

by   Yazhan Zhang, et al.

In essence, successful grasp boils down to correct responses to multiple contact events between fingertips and objects. In most scenarios, tactile sensing is adequate to distinguish contact events. Due to the nature of high dimensionality of tactile information, classifying spatiotemporal tactile signals using conventional model-based methods is difficult. In this work, we propose to predict and classify tactile signal using deep learning methods, seeking to enhance the adaptability of the robotic grasp system to external event changes that may lead to grasping failure. We develop a deep learning framework and collect 6650 tactile image sequences with a vision-based tactile sensor, and the neural network is integrated into a contact-event-based robotic grasping system. In grasping experiments, we achieved 52 object lifting success rate with contact detection, significantly higher robustness under unexpected loads with slip prediction compared with open-loop grasps, demonstrating that integration of the proposed framework into robotic grasping system substantially improves picking success rate and capability to withstand external disturbances.


page 2

page 4

page 5

page 7

page 8


The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?

A successful grasp requires careful balancing of the contact forces. Ded...

Tactile Grasp Refinement using Deep Reinforcement Learning and Analytic Grasp Stability Metrics

Reward functions are at the heart of every reinforcement learning (RL) a...

MAT: Multi-Fingered Adaptive Tactile Grasping via Deep Reinforcement Learning

Vision-based grasping systems typically adopt an open-loop execution of ...

Where Shall I Touch? Vision-Guided Tactile Poking for Transparent Object Grasping

Picking up transparent objects is still a challenging task for robots. T...

Interpreting and Predicting Tactile Signals for the SynTouch BioTac

In the human hand, high-density contact information provided by afferent...

Exploiting High Quality Tactile Sensors for Simplified Grasping

Robots are expected to grasp a wide range of objects varying in shape, w...