Classify, predict, detect, anticipate and synthesize: Hierarchical recurrent latent variable models for human activity modeling

09/24/2018
by   Judith Butepage, et al.
0

Human activity modeling operates on two levels: high-level action modeling, such as classification, prediction, detection and anticipation, and low-level motion trajectory prediction and synthesis. In this work, we propose a semi-supervised generative latent variable model that addresses both of these levels by modeling continuous observations as well as semantic labels. We extend the model to capture the dependencies between different entities, such as a human and objects, and to represent hierarchical label structure, such as high-level actions and sub-activities. In the experiments we investigate our model's capability to classify, predict, detect and anticipate semantic action and affordance labels and to predict and generate human motion. We train our models on data extracted from depth image streams from the Cornell Activity 120, the UTKinect-Action3D and the Stony Brook University Kinect Interaction Dataset. We observe that our model performs well in all of the tasks and often outperforms task-specific models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/19/2018

Detect, anticipate and generate: Semi-supervised recurrent latent variable models for human activity modeling

Successful Human-Robot collaboration requires a predictive model of huma...
research
11/01/2016

Learning recurrent representations for hierarchical behavior modeling

We propose a framework for detecting action patterns from motion sequenc...
research
07/11/2022

Hierarchical Latent Structure for Multi-Modal Vehicle Trajectory Forecasting

Variational autoencoder (VAE) has widely been utilized for modeling data...
research
08/20/2020

Learning to Abstract and Predict Human Actions

Human activities are naturally structured as hierarchies unrolled over t...
research
03/07/2023

Speech Modeling with a Hierarchical Transformer Dynamical VAE

The dynamical variational autoencoders (DVAEs) are a family of latent-va...
research
07/12/2022

A semi-supervised geometric-driven methodology for supervised fishing activity detection on multi-source AIS tracking messages

Automatic Identification System (AIS) messages are useful for tracking v...
research
07/28/2017

Recurrent Ladder Networks

We propose a recurrent extension of the Ladder networks whose structure ...

Please sign up or login with your details

Forgot password? Click here to reset