Grounding Graph Network Simulators using Physical Sensor Observations

by   Jonas Linkerhägner, et al.

Physical simulations that accurately model reality are crucial for many engineering disciplines such as mechanical engineering and robotic motion planning. In recent years, learned Graph Network Simulators produced accurate mesh-based simulations while requiring only a fraction of the computational cost of traditional simulators. Yet, the resulting predictors are confined to learning from data generated by existing mesh-based simulators and thus cannot include real world sensory information such as point cloud data. As these predictors have to simulate complex physical systems from only an initial state, they exhibit a high error accumulation for long-term predictions. In this work, we integrate sensory information to ground Graph Network Simulators on real world observations. In particular, we predict the mesh state of deformable objects by utilizing point cloud data. The resulting model allows for accurate predictions over longer time horizons, even under uncertainties in the simulation, such as unknown material properties. Since point clouds are usually not available for every time step, especially in online settings, we employ an imputation-based model. The model can make use of such additional information only when provided, and resorts to a standard Graph Network Simulator, otherwise. We experimentally validate our approach on a suite of prediction tasks for mesh-based interactions between soft and rigid bodies. Our method results in utilization of additional point cloud information to accurately predict stable simulations where existing Graph Network Simulators fail.


page 2

page 6

page 7

page 19

page 20

page 21

page 22

page 26


Learning Mesh-Based Simulation with Graph Networks

Mesh-based simulations are central to modeling complex physical systems ...

3D Adversarial Attacks Beyond Point Cloud

Previous adversarial attacks on 3D point clouds mainly focus on add pert...

AGAR: Attention Graph-RNN for Adaptative Motion Prediction of Point Clouds of Deformable Objects

This paper focuses on motion prediction for point cloud sequences in the...

Practical Shape Analysis and Segmentation Methods for Point Cloud Models

Current point cloud processing algorithms do not have the capability to ...

Learning A Physical Long-term Predictor

Evolution has resulted in highly developed abilities in many natural int...

Accurately Solving Physical Systems with Graph Learning

Iterative solvers are widely used to accurately simulate physical system...

A Motion Planning Strategy for the Active Vision-Based Mapping of Ground-Level Structures

This paper presents a strategy to guide a mobile ground robot equipped w...

Please sign up or login with your details

Forgot password? Click here to reset