Variational Tracking and Prediction with Generative Disentangled State-Space Models

by   Adnan Akhundov, et al.

We address tracking and prediction of multiple moving objects in visual data streams as inference and sampling in a disentangled latent state-space model. By encoding objects separately and including explicit position information in the latent state space, we perform tracking via amortized variational Bayesian inference of the respective latent positions. Inference is implemented in a modular neural framework tailored towards our disentangled latent space. Generative and inference model are jointly learned from observations only. Comparing to related prior work, we empirically show that our Markovian state-space assumption enables faithful and much improved long-term prediction well beyond the training horizon. Further, our inference model correctly decomposes frames into objects, even in the presence of occlusions. Tracking performance is increased significantly over prior art.


page 2

page 4

page 6

page 21

page 22

page 23

page 24

page 25


Linear Variational State Space Filtering

We introduce Variational State-Space Filters (VSSF), a new method for un...

Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data

We introduce Deep Variational Bayes Filters (DVBF), a new method for uns...

Comparing the latent space of generative models

Different encodings of datapoints in the latent space of latent-vector g...

Recency-weighted Markovian inference

We describe a Markov latent state space (MLSS) model, where the latent s...

Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs

We present a simple neural rendering architecture that helps variational...

Factored Latent Analysis for far-field tracking data

This paper uses Factored Latent Analysis (FLA) to learn a factorized, se...

Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoF

We solve the problem of 6-DoF localisation and 3D dense reconstruction i...