Which priors matter? Benchmarking models for learning latent dynamics

11/09/2021
by   Aleksandar Botev, et al.
0

Learning dynamics is at the heart of many important applications of machine learning (ML), such as robotics and autonomous driving. In these settings, ML algorithms typically need to reason about a physical system using high dimensional observations, such as images, without access to the underlying state. Recently, several methods have proposed to integrate priors from classical mechanics into ML models to address the challenge of physical reasoning from images. In this work, we take a sober look at the current capabilities of these models. To this end, we introduce a suite consisting of 17 datasets with visual observations based on physical systems exhibiting a wide range of dynamics. We conduct a thorough and detailed comparison of the major classes of physically inspired methods alongside several strong baselines. While models that incorporate physical priors can often learn latent spaces with desirable properties, our results demonstrate that these methods fail to significantly improve upon standard techniques. Nonetheless, we find that the use of continuous and time-reversible dynamics benefits models of all classes.

READ FULL TEXT

page 5

page 6

research
11/10/2021

SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Vision

A recently proposed class of models attempts to learn latent dynamics fr...
research
12/07/2020

Variational Autoencoders for Learning Nonlinear Dynamics of Physical Systems

We develop data-driven methods for incorporating physical information fo...
research
06/25/2019

A unified sparse optimization framework to learn parsimonious physics-informed models from data

Machine learning (ML) is redefining what is possible in data-intensive f...
research
06/05/2019

Machine Learning and System Identification for Estimation in Physical Systems

In this thesis, we draw inspiration from both classical system identific...
research
02/14/2019

Tractography and machine learning: Current state and open challenges

Supervised machine learning (ML) algorithms have recently been proposed ...
research
06/05/2017

Visual Interaction Networks

From just a glance, humans can make rich predictions about the future st...
research
11/24/2022

Learning Integrable Dynamics with Action-Angle Networks

Machine learning has become increasingly popular for efficiently modelli...

Please sign up or login with your details

Forgot password? Click here to reset