More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch

05/28/2018
by   Roberto Calandra, et al.
6

For humans, the process of grasping an object relies heavily on rich tactile feedback. Recent robotic grasping work, however, has been largely based only on visual input, and thus cannot easily benefit from feedback after initiating contact. In this paper, we investigate if a robot can learn to use tactile information to iteratively and efficiently adjust its grasp. To this end, we propose an end-to-end action-conditional model that learns greedy regrasping policies from raw visuo-tactile data. This model - a deep, multimodal convolutional network - predicts the outcome of a candidate grasp adjustment, and then executes a grasp by iteratively selecting the most promising actions. Our approach requires neither calibration of the tactile sensors, nor any analytical modeling of contact forces, thus reducing the engineering effort required to obtain efficient grasping policies. We train our model with data from over 6,450 grasping trials on a two-finger gripper equipped with GelSight high-resolution tactile sensors on each finger. Across extensive experiments, our approach outperforms a variety of baselines at (i) estimating grasp adjustment outcomes, (ii) selecting efficient grasp adjustments for quick grasping, and (iii) reducing the amount of force applied at the fingers, while maintaining competitive performance. Finally, we study the choices made by our model and show that it has successfully acquired useful and interpretable grasping behaviors.

READ FULL TEXT

page 1

page 2

page 4

page 6

page 7

research
10/16/2017

The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?

A successful grasp requires careful balancing of the contact forces. Ded...
research
05/10/2018

Learning to Grasp Without Seeing

Can a robot grasp an unknown object without seeing it? In this paper, we...
research
03/16/2023

Tactile-Driven Gentle Grasping for Human-Robot Collaborative Tasks

This paper presents a control scheme for force sensitive, gentle graspin...
research
07/25/2022

Exploiting High Quality Tactile Sensors for Simplified Grasping

Robots are expected to grasp a wide range of objects varying in shape, w...
research
09/23/2021

Tactile Grasp Refinement using Deep Reinforcement Learning and Analytic Grasp Stability Metrics

Reward functions are at the heart of every reinforcement learning (RL) a...
research
07/14/2021

Sensorimotor-inspired Tactile Feedback and Control Improve Consistency of Prosthesis Manipulation in the Absence of Direct Vision

The lack of haptically aware upper-limb prostheses forces amputees to re...
research
04/13/2022

Single-grasp deformable object discrimination: the effect of gripper morphology, sensing modalities, and action parameters

We studied the discrimination of deformable objects by grasping them usi...

Please sign up or login with your details

Forgot password? Click here to reset