Vid2Param: Online system identification from video for robotics applications

07/15/2019
by   Martin Asenov, et al.
0

Robots performing tasks in dynamic environments would benefit greatly from understanding the underlying environment motion, in order to make future predictions and to synthesize effective control policies that use this inductive bias. Online system identification is therefore a fundamental requirement for robust autonomous agents. When the dynamics involves multiple modes (due to contacts or interactions between objects), and when system identification must proceed directly from a rich sensory stream such as video, then traditional methods for system identification may not be well suited. We propose an approach wherein fast parameter estimation with a model can be seamlessly combined with a recurrent variational autoencoder. Our Physics-based recurrent variational autoencoder model includes an additional loss that enforces conformity with the structure of a physically based dynamics model. This enables the resulting model to encode parameters such as position, velocity, restitution, air drag and other physical properties of the system. The model can be trained entirely in simulation, in an end-to-end manner with domain randomization, to perform online system identification, and probabilistic forward predictions of parameters of interest. We benchmark against existing system identification methods and demonstrate that Vid2Param outperforms the baselines in terms of speed and accuracy of identification, and also provides uncertainty quantification in the form of a distribution over future trajectories. Furthermore, we illustrate the utility of this in physical experiments wherein a PR2 robot with velocity constrained arm must intercept a bouncing ball, by estimating the physical parameters of this ball directly from the video trace after the ball is released.

READ FULL TEXT

page 4

page 7

page 8

research
05/15/2023

Physics-enhanced Gaussian Process Variational Autoencoder

Variational autoencoders allow to learn a lower-dimensional latent space...
research
11/15/2022

Autonomous Golf Putting with Data-Driven and Physics-Based Methods

We are developing a self-learning mechatronic golf robot using combined ...
research
05/27/2019

Physics-as-Inverse-Graphics: Joint Unsupervised Learning of Objects and Physics from Video

We aim to perform unsupervised discovery of objects and their states suc...
research
09/13/2018

Semiparametrical Gaussian Processes Learning of Forward Dynamical Models for Navigating in a Circular Maze

This paper presents a problem of model learning for the purpose of learn...
research
11/05/2020

STReSSD: Sim-To-Real from Sound for Stochastic Dynamics

Sound is an information-rich medium that captures dynamic physical event...
research
06/04/2019

BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators

We introduce BayesSim, a framework for robotics simulations allowing a f...
research
03/19/2021

Joint Parameter Discovery and Generative Modeling of Dynamic Systems

Given an unknown dynamic system such as a coupled harmonic oscillator wi...

Please sign up or login with your details

Forgot password? Click here to reset