Model-free vision-based shaping of deformable plastic materials

by   Andrea Cherubini, et al.

We address the problem of shaping deformable plastic materials using non-prehensile actions. Shaping plastic objects is challenging, since they are difficult to model and to track visually. We study this problem, by using kinetic sand, a plastic toy material which mimics the physical properties of wet sand. Inspired by a pilot study where humans shape kinetic sand, we define two types of actions: pushing the material from the sides and tapping from above. The chosen actions are executed with a robotic arm using image-based visual servoing. From the current and desired view of the material, we define states based on visual features such as the outer contour shape and the pixel luminosity values. These are mapped to actions, which are repeated iteratively to reduce the image error until convergence is reached. For pushing, we propose three methods for mapping the visual state to an action. These include heuristic methods and a neural network, trained from human actions. We show that it is possible to obtain simple shapes with the kinetic sand, without explicitly modeling the material. Our approach is limited in the types of shapes it can achieve. A richer set of action types and multi-step reasoning is needed to achieve more sophisticated shapes.



page 2

page 4

page 6

page 8

page 12

page 13

page 14

page 15


Manipulating Highly Deformable Materials Using a Visual Feedback Dictionary

The complex physical properties of highly deformable materials like clot...

Mesh2Fab: Reforming Shapes for Material-specific Fabrication

As humans, we regularly associate shape of an object with its built mate...

Learning an Action-Conditional Model for Haptic Texture Generation

Rich haptic sensory feedback in response to user interactions is desirab...

Recognizing Material Properties from Images

Humans rely on properties of the materials that make up objects to guide...

Physics-based Mesh Deformation with Haptic Feedback and Material Anisotropy

We present a physics-based framework to simulate porous, deformable mate...

Tool Substitution with Shape and Material Reasoning Using Dual Neural Networks

This paper explores the problem of tool substitution, namely, identifyin...

Visual Reaction: Learning to Play Catch with Your Drone

In this paper we address the problem of visual reaction: the task of int...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.