DeepAI AI Chat
Log In Sign Up

Reasoning About Physical Interactions with Object-Oriented Prediction and Planning

by   Michael Janner, et al.
berkeley college

Object-based factorizations provide a useful level of abstraction for interacting with the world. Building explicit object representations, however, often requires supervisory signals that are difficult to obtain in practice. We present a paradigm for learning object-centric representations for physical scene understanding without direct supervision of object properties. Our model, Object-Oriented Prediction and Planning (O2P2), jointly learns a perception function to map from image observations to object representations, a pairwise physics interaction function to predict the time evolution of a collection of objects, and a rendering function to map objects back to pixels. For evaluation, we consider not only the accuracy of the physical predictions of the model, but also its utility for downstream tasks that require an actionable representation of intuitive physics. After training our model on an image prediction task, we can use its learned representations to build block towers more complicated than those observed during training.


page 1

page 2

page 5

page 6

page 7

page 8

page 12


DensePhysNet: Learning Dense Physical Object Representations via Multi-step Dynamic Interactions

We study the problem of learning physical object representations for rob...

Entity Abstraction in Visual Model-Based Reinforcement Learning

This paper tests the hypothesis that modeling a scene in terms of entiti...

KINet: Keypoint Interaction Networks for Unsupervised Forward Modeling

Object-centric representation is an essential abstraction for physical r...

Physical Primitive Decomposition

Objects are made of parts, each with distinct geometry, physics, functio...

Solving Physics Puzzles by Reasoning about Paths

We propose a new deep learning model for goal-driven tasks that require ...

Unsupervised Intuitive Physics from Past Experiences

We are interested in learning models of intuitive physics similar to the...

Graph networks as learnable physics engines for inference and control

Understanding and interacting with everyday physical scenes requires ric...