Predicting the dynamics of 2d objects with a deep residual network

10/13/2016
by   François Fleuret, et al.
0

We investigate how a residual network can learn to predict the dynamics of interacting shapes purely as an image-to-image regression task. With a simple 2d physics simulator, we generate short sequences composed of rectangles put in motion by applying a pulling force at a point picked at random. The network is trained with a quadratic loss to predict the image of the resulting configuration, given the image of the starting configuration and an image indicating the point of grasping. Experiments show that the network learns to predict accurately the resulting image, which implies in particular that (1) it segments rectangles as distinct components, (2) it infers which one contains the grasping point, (3) it models properly the dynamic of a single rectangle, including the torque, (4) it detects and handles collisions to some extent, and (5) it re-synthesizes properly the entire scene with displaced rectangles.

READ FULL TEXT
research
01/16/2020

Predicting Target Feature Configuration of Non-stationary Objects for Grasping with Image-Based Visual Servoing

In this paper we consider the problem of the final approach stage of clo...
research
03/27/2019

TossingBot: Learning to Throw Arbitrary Objects with Residual Physics

We investigate whether a robot arm can learn to pick and throw arbitrary...
research
07/19/2021

Towards synthesizing grasps for 3D deformable objects with physics-based simulation

Grasping deformable objects is not well researched due to the complexity...
research
09/11/2019

Antipodal Robotic Grasping using Generative Residual Convolutional Neural Network

In this paper, we tackle the problem of generating antipodal robotic gra...
research
01/16/2020

Predicting the Physical Dynamics of Unseen 3D Objects

Machines that can predict the effect of physical interactions on the dyn...
research
06/15/2019

Deep Set Prediction Networks

We study the problem of predicting a set from a feature vector with a de...
research
01/04/2021

A Pushing-Grasping Collaborative Method Based on Deep Q-Network Algorithm in Dual Perspectives

Aiming at the traditional grasping method for manipulators based on 2D c...

Please sign up or login with your details

Forgot password? Click here to reset