Classification of sparsely labeled spatio-temporal data through semi-supervised adversarial learning

01/26/2018
by   Atanas Mirchev, et al.
0

In recent years, Generative Adversarial Networks (GAN) have emerged as a powerful method for learning the mapping from noisy latent spaces to realistic data samples in high-dimensional space. So far, the development and application of GANs have been predominantly focused on spatial data such as images. In this project, we aim at modeling of spatio-temporal sensor data instead, i.e. dynamic data over time. The main goal is to encode temporal data into a global and low-dimensional latent vector that captures the dynamics of the spatio-temporal signal. To this end, we incorporate auto-regressive RNNs, Wasserstein GAN loss, spectral norm weight constraints and a semi-supervised learning scheme into InfoGAN, a method for retrieval of meaningful latents in adversarial learning. To demonstrate the modeling capability of our method, we encode full-body skeletal human motion from a large dataset representing 60 classes of daily activities, recorded in a multi-Kinect setup. Initial results indicate competitive classification performance of the learned latent representations, compared to direct CNN/RNN inference. In future work, we plan to apply this method on a related problem in the medical domain, i.e. on recovery of meaningful latents in gait analysis of patients with vertigo and balance disorders.

READ FULL TEXT
research
08/18/2020

Generative Adversarial Networks for Spatio-temporal Data: A Survey

Generative Adversarial Networks (GANs) have shown remarkable success in ...
research
12/07/2017

Semi-Supervised Learning with IPM-based GANs: an Empirical Study

We present an empirical investigation of a recent class of Generative Ad...
research
10/01/2021

SMATE: Semi-Supervised Spatio-Temporal Representation Learning on Multivariate Time Series

Learning from Multivariate Time Series (MTS) has attracted widespread at...
research
05/19/2019

Spatio-Temporal Adversarial Learning for Detecting Unseen Falls

Fall detection is an important problem from both the health and machine ...
research
10/29/2018

Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning

We address the problem of segmenting 3D multi-modal medical images in sc...
research
05/19/2014

Kronecker PCA Based Spatio-Temporal Modeling of Video for Dismount Classification

We consider the application of KronPCA spatio-temporal modeling techniqu...
research
03/26/2020

Gaussian-Dirichlet Random Fields for Inference over High Dimensional Categorical Observations

We propose a generative model for the spatio-temporal distribution of hi...

Please sign up or login with your details

Forgot password? Click here to reset