The Surprising Effectiveness of Linear Models for Visual Foresight in Object Pile Manipulation

02/21/2020
by   H. J. Terry Suh, et al.
0

In this paper, we tackle the problem of pushing piles of small objects into a desired target set using visual feedback. Unlike conventional single-object manipulation pipelines, which estimate the state of the system parametrized by pose, the underlying physical state of this system is difficult to observe from images. Thus, we take the approach of reasoning directly in the space of images, and acquire the dynamics of visual measurements in order to synthesize a visual-feedback policy. We present a simple controller using an image-space Lyapunov function, and evaluate the closed-loop performance using three different class of models for image prediction: deep-learning-based models for image-to-image translation, an object-centric model obtained from treating each pixel as a particle, and a switched-linear system where an action-dependent linear map is used. Through results in simulation and experiment, we show that for this task, a linear model works surprisingly well – achieving better prediction error, downstream task performance, and generalization to new environments than the deep models we trained on the same amount of data. We believe these results provide an interesting example in the spectrum of models that are most useful for vision-based feedback in manipulation, considering both the quality of visual prediction, as well as compatibility with rigorous methods for control design and analysis.

READ FULL TEXT

page 2

page 8

page 10

page 11

page 12

page 13

page 14

research
07/16/2020

Model-Based Manipulation of Linear Flexible Objects with Visual Curvature Feedback

Manipulation of deformable objects is a desired skill in making robots u...
research
10/20/2022

VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors

We introduce VIOLA, an object-centric imitation learning approach to lea...
research
11/14/2019

Self-Supervised Learning of State Estimation for Manipulating Deformable Linear Objects

We demonstrate model-based, visual robot manipulation of linear deformab...
research
10/27/2020

Transporter Networks: Rearranging the Visual World for Robotic Manipulation

Robotic manipulation can be formulated as inducing a sequence of spatial...
research
10/05/2022

On the Learning Mechanisms in Physical Reasoning

Is dynamics prediction indispensable for physical reasoning? If so, what...
research
08/02/2021

An Efficient Image-to-Image Translation HourGlass-based Architecture for Object Pushing Policy Learning

Humans effortlessly solve pushing tasks in everyday life but unlocking t...
research
11/04/2021

Deep Direct Visual Servoing of Tendon-Driven Continuum Robots

Vision-based control has found a key place in the research to tackle the...

Please sign up or login with your details

Forgot password? Click here to reset