Taxim: An Example-based Simulation Model for GelSight Tactile Sensors

by   Zilin Si, et al.

Simulation is widely used in robotics for system verification and large-scale data collection. However, simulating sensors, including tactile sensors, has been a long-standing challenge. In this paper, we propose Taxim, a realistic and high-speed simulation model for a vision-based tactile sensor, GelSight. A GelSight sensor uses a piece of soft elastomer as the medium of contact and embeds optical structures to capture the deformation of the elastomer, which infers the geometry and forces applied at the contact surface. We propose an example-based method for simulating GelSight: we simulate the optical response to the deformation with a polynomial look-up table. This table maps the deformed geometries to pixel intensity sampled by the embedded camera. In order to simulate the surface markers' motion that is caused by the surface stretch of the elastomer, we apply the linear elastic deformation theory and the superposition principle. The simulation model is calibrated with less than 100 data points from a real sensor. The example-based approach enables the model to easily migrate to other GelSight sensors or its variations. To the best of our knowledge, our simulation framework is the first to incorporate marker motion field simulation that derives from elastomer deformation together with the optical simulation, creating a comprehensive and computationally efficient tactile simulation framework. Experiments reveal that our optical simulation has the lowest pixel-wise intensity errors compared to prior work and can run online with CPU computing.


page 1

page 3

page 5

page 6

page 8


Beyond Flat GelSight Sensors: Simulation of Optical Tactile Sensors of Complex Morphologies for Sim2Real Learning

Recently, several morphologies, each with its advantages, have been prop...

Tacchi: A Pluggable and Low Computational Cost Elastomer Deformation Simulator for Optical Tactile Sensors

Simulation is widely applied in robotics research to save time and resou...

Learning to Read Braille: Bridging the Tactile Reality Gap with Diffusion Models

Simulating vision-based tactile sensors enables learning models for cont...

WSTac: Interactive Surface Perception based on Whisker-Inspired and Self-Illuminated Vision-Based Tactile Sensor

Modern Visual-Based Tactile Sensors (VBTSs) use cost-effective cameras t...

Deformation Capture via Soft and Stretchable Sensor Arrays

We propose a hardware and software pipeline to fabricate flexible wearab...

Highly curved image sensors: a practical approach for improved optical performance

The significant optical and size benefits of using a curved focal surfac...

EMU: Efficient Muscle Simulation In Deformation Space

EMU is an efficient and scalable model to simulate bulk musculoskeletal ...

Please sign up or login with your details

Forgot password? Click here to reset