Flexible Neural Representation for Physics Prediction

06/21/2018
by   Damian Mrowca, et al.
2

Humans have a remarkable capacity to understand the physical dynamics of objects in their environment, flexibly capturing complex structures and interactions at multiple levels of detail. Inspired by this ability, we propose a hierarchical particle-based object representation that covers a wide variety of types of three-dimensional objects, including both arbitrary rigid geometrical shapes and deformable materials. We then describe the Hierarchical Relation Network (HRN), an end-to-end differentiable neural network based on hierarchical graph convolution, that learns to predict physical dynamics in this representation. Compared to other neural network baselines, the HRN accurately handles complex collisions and nonrigid deformations, generating plausible dynamics predictions at long time scales in novel settings, and scaling to large scene configurations. These results demonstrate an architecture with the potential to form the basis of next-generation physics predictors for use in computer vision, robotics, and quantitative cognitive science.

READ FULL TEXT

page 2

page 3

page 7

page 14

page 15

page 17

page 19

page 21

research
12/01/2016

A Compositional Object-Based Approach to Learning Physical Dynamics

We present the Neural Physics Engine (NPE), a framework for learning sim...
research
01/02/2019

Learning Generalizable Physical Dynamics of 3D Rigid Objects

Humans have a remarkable ability to predict the effect of physical inter...
research
04/28/2020

Visual Grounding of Learned Physical Models

Humans intuitively recognize objects' physical properties and predict th...
research
09/07/2018

Neural Allocentric Intuitive Physics Prediction from Real Videos

Humans are able to make rich predictions about the future dynamics of ph...
research
09/05/2018

Modeling human intuitions about liquid flow with particle-based simulation

Humans can easily describe, imagine, and, crucially, predict a wide vari...
research
03/04/2021

Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects

Capturing scene dynamics and predicting the future scene state is challe...
research
04/15/2019

Bounce and Learn: Modeling Scene Dynamics with Real-World Bounces

We introduce an approach to model surface properties governing bounces i...

Please sign up or login with your details

Forgot password? Click here to reset