TACTO: A Fast, Flexible and Open-source Simulator for High-Resolution Vision-based Tactile Sensors

12/15/2020
by   Shaoxiong Wang, et al.
16

Simulators perform an important role in prototyping, debugging and benchmarking new advances in robotics and learning for control. Although many physics engines exist, some aspects of the real-world are harder than others to simulate. One of the aspects that have so far eluded accurate simulation is touch sensing. To address this gap, we present TACTO – a fast, flexible and open-source simulator for vision-based tactile sensors. This simulator allows to render realistic high-resolution touch readings at hundreds of frames per second, and can be easily configured to simulate different vision-based tactile sensors, including GelSight, DIGIT and OmniTact. In this paper, we detail the principles that drove the implementation of TACTO and how they are reflected in its architecture. We demonstrate TACTO on a perceptual task, by learning to predict grasp stability using touch from 1 million grasps, and on a marble manipulation control task. We believe that TACTO is a step towards the widespread adoption of touch sensing in robotic applications, and to enable machine learning practitioners interested in multi-modal learning and control. TACTO is open-source at https://github.com/facebookresearch/tacto.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

research
12/24/2020

Simulation of Vision-based Tactile Sensors using Physics based Rendering

Tactile sensing has seen a rapid adoption with the advent of vision-base...
research
05/29/2020

DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation

Despite decades of research, general purpose in-hand manipulation remain...
research
05/26/2021

PyTouch: A Machine Learning Library for Touch Processing

With the increased availability of rich tactile sensors, there is an equ...
research
06/12/2022

Integrating High-Resolution Tactile Sensing into Grasp Stability Prediction

We investigate how high-resolution tactile sensors can be utilized in co...
research
12/07/2018

Transfer learning for vision-based tactile sensing

Due to the complexity of modeling the elastic properties of materials, t...
research
04/03/2023

Learning to Read Braille: Bridging the Tactile Reality Gap with Diffusion Models

Simulating vision-based tactile sensors enables learning models for cont...

Please sign up or login with your details

Forgot password? Click here to reset