Bounce and Learn: Modeling Scene Dynamics with Real-World Bounces

by   Senthil Purushwalkam, et al.

We introduce an approach to model surface properties governing bounces in everyday scenes. Our model learns end-to-end, starting from sensor inputs, to predict post-bounce trajectories and infer two underlying physical properties that govern bouncing - restitution and effective collision normals. Our model, Bounce and Learn, comprises two modules -- a Physics Inference Module (PIM) and a Visual Inference Module (VIM). VIM learns to infer physical parameters for locations in a scene given a single still image, while PIM learns to model physical interactions for the prediction task given physical parameters and observed pre-collision 3D trajectories. To achieve our results, we introduce the Bounce Dataset comprising 5K RGB-D videos of bouncing trajectories of a foam ball to probe surfaces of varying shapes and materials in everyday scenes including homes and offices. Our proposed model learns from our collected dataset of real-world bounces and is bootstrapped with additional information from simple physics simulations. We show on our newly collected dataset that our model out-performs baselines, including trajectory fitting with Newtonian physics, in predicting post-bounce trajectories and inferring physical properties of a scene.


page 2

page 7

page 8

page 13

page 18

page 20

page 21

page 22


Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language

In this work, we propose a unified framework, called Visual Reasoning wi...

ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queries

Existing autonomous driving pipelines separate the perception module fro...

Learning Manipulation under Physics Constraints with Visual Perception

Understanding physical phenomena is a key competence that enables humans...

Visual Interaction Networks

From just a glance, humans can make rich predictions about the future st...

Flexible Neural Representation for Physics Prediction

Humans have a remarkable capacity to understand the physical dynamics of...

SMASH: Physics-guided Reconstruction of Collisions from Videos

Collision sequences are commonly used in games and entertainment to add ...

Please sign up or login with your details

Forgot password? Click here to reset