Combining learned and analytical models for predicting action effects

by   Alina Kloss, et al.

One of the most basic skills a robot should possess is predicting the effect of physical interactions with objects in the environment. This enables optimal action selection to reach a certain goal state. Traditionally, these dynamics are described by physics-based analytical models, which may however be very hard to find for complex problems. More recently, we have seen learning approaches that can predict the effect of more complex physical interactions directly from sensory input. However, it is an open question how far these models generalize beyond their training data. In this work, we analyse how analytical and learned models can be combined to leverage the best of both worlds. As physical interaction task, we use planar pushing, for which there exists a well-known analytical model and a large real-world dataset. We propose to use a neural network to convert the raw sensory data into a suitable representation that can be consumed by the analytical model and compare this approach to using neural networks for both, perception and prediction. Our results show that the combined method outperforms the purely learned version in terms of accuracy and generalization to push actions not seen during training. It also performs comparable to the analytical model applied on ground truth input values, despite using raw sensory data as input.


page 5

page 8

page 9


Online Learning in Planar Pushing with Combined Prediction Model

Pushing is a useful robotic capability for positioning and reorienting o...

Combining Physical Simulators and Object-Based Networks for Control

Physics engines play an important role in robot planning and control; ho...

Predicting the Physical Dynamics of Unseen 3D Objects

Machines that can predict the effect of physical interactions on the dyn...

Augmenting Physical Simulators with Stochastic Neural Networks: Case Study of Planar Pushing and Bouncing

An efficient, generalizable physical simulator with universal uncertaint...

Trajectory Optimization Using Neural Network Gradients of Learned Dynamics

Trajectory optimization methods have achieved an exceptional level of pe...

PIP: Physical Interaction Prediction via Mental Imagery with Span Selection

To align advanced artificial intelligence (AI) with human values and pro...

Taking Visual Motion Prediction To New Heightfields

While the basic laws of Newtonian mechanics are well understood, explain...