Deep Tactile Experience: Estimating Tactile Sensor Output from Depth Sensor Data

10/17/2021
by   Karankumar Patel, et al.
0

Tactile sensing is inherently contact based. To use tactile data, robots need to make contact with the surface of an object. This is inefficient in applications where an agent needs to make a decision between multiple alternatives that depend the physical properties of the contact location. We propose a method to get tactile data in a non-invasive manner. The proposed method estimates the output of a tactile sensor from the depth data of the surface of the object based on past experiences. An experience dataset is built by allowing the robot to interact with various objects, collecting tactile data and the corresponding object surface depth data. We use the experience dataset to train a neural network to estimate the tactile output from depth data alone. We use GelSight tactile sensors, an image-based sensor, to generate images that capture detailed surface features at the contact location. We train a network with a dataset containing 578 tactile-image to depthmap correspondences. Given a depth-map of the surface of an object, the network outputs an estimate of the response of the tactile sensor, should it make a contact with the object. We evaluate the method with structural similarity index matrix (SSIM), a similarity metric between two images commonly used in image processing community. We present experimental results that show the proposed method outperforms a baseline that uses random images with statistical significance getting an SSIM score of 0.84 +/- 0.0056 and 0.80 +/- 0.0036, respectively.

READ FULL TEXT

page 1

page 3

page 5

page 6

research
02/28/2021

TouchRoller: A Rolling Optical Tactile Sensor for Rapid Assessment of Large Surfaces

Tactile sensing is important for robots to perceive the world as it capt...
research
02/06/2023

Learning to Predict Grip Quality from Simulation: Establishing a Digital Twin to Generate Simulated Data for a Grip Stability Metric

A robust grip is key to successful manipulation and joining of work piec...
research
05/21/2018

Voronoi Features for Tactile Sensing: Direct Inference of Pressure, Shear, and Contact Locations

There are a wide range of features that tactile contact provides, each w...
research
07/26/2023

Attention for Robot Touch: Tactile Saliency Prediction for Robust Sim-to-Real Tactile Control

High-resolution tactile sensing can provide accurate information about l...
research
08/16/2020

A Biomimetic Tactile Fingerprint Induces Incipient Slip

We present a modified TacTip biomimetic optical tactile sensor design wh...
research
05/04/2023

Controllable Visual-Tactile Synthesis

Deep generative models have various content creation applications such a...
research
12/02/2019

Surface Following using Deep Reinforcement Learning and a GelSightTactile Sensor

Tactile sensors can provide detailed contact in-formation that can facil...

Please sign up or login with your details

Forgot password? Click here to reset