Hamiltonian prior to Disentangle Content and Motion in Image Sequences

12/02/2021
by   Asif Khan, et al.
5

We present a deep latent variable model for high dimensional sequential data. Our model factorises the latent space into content and motion variables. To model the diverse dynamics, we split the motion space into subspaces, and introduce a unique Hamiltonian operator for each subspace. The Hamiltonian formulation provides reversible dynamics that learn to constrain the motion path to conserve invariant properties. The explicit split of the motion space decomposes the Hamiltonian into symmetry groups and gives long-term separability of the dynamics. This split also means representations can be learnt that are easy to interpret and control. We demonstrate the utility of our model for swapping the motion of two videos, generating sequences of various actions from a given image and unconditional sequence generation.

READ FULL TEXT

page 8

page 9

page 16

page 17

page 18

page 19

research
03/08/2018

A Deep Generative Model for Disentangled Representations of Sequential Data

We present a VAE architecture for encoding and generating high dimension...
research
11/25/2017

Hamiltonian Path in Split Graphs- a Dichotomy

In this paper, we investigate Hamiltonian path problem in the context of...
research
11/10/2021

SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Vision

A recently proposed class of models attempts to learn latent dynamics fr...
research
09/23/2022

Learning Interpretable Dynamics from Images of a Freely Rotating 3D Rigid Body

In many real-world settings, image observations of freely rotating 3D ri...
research
08/02/2023

Data-Driven Identification of Quadratic Symplectic Representations of Nonlinear Hamiltonian Systems

We present a framework for learning Hamiltonian systems using data. This...
research
08/22/2022

Constants of motion network

The beauty of physics is that there is usually a conserved quantity in a...
research
08/22/2023

Hamiltonian GAN

A growing body of work leverages the Hamiltonian formalism as an inducti...

Please sign up or login with your details

Forgot password? Click here to reset