Fine-grained Action Segmentation using the Semi-Supervised Action GAN

09/20/2019
by   Harshala Gammulle, et al.
18

In this paper we address the problem of continuous fine-grained action segmentation, in which multiple actions are present in an unsegmented video stream. The challenge for this task lies in the need to represent the hierarchical nature of the actions and to detect the transitions between actions, allowing us to localise the actions within the video effectively. We propose a novel recurrent semi-supervised Generative Adversarial Network (GAN) model for continuous fine-grained human action segmentation. Temporal context information is captured via a novel Gated Context Extractor (GCE) module, composed of gated attention units, that directs the queued context information through the generator model, for enhanced action segmentation. The GAN is made to learn features in a semi-supervised manner, enabling the model to perform action classification jointly with the standard, unsupervised, GAN learning procedure. We perform extensive evaluations on different architectural variants to demonstrate the importance of the proposed network architecture, and show that it is capable of outperforming current state-of-the-art on three challenging datasets: 50 Salads, MERL Shopping and Georgia Tech Egocentric Activities dataset.

READ FULL TEXT

page 18

page 20

page 27

page 28

research
09/20/2019

Coupled Generative Adversarial Network for Continuous Fine-grained Action Segmentation

We propose a novel conditional GAN (cGAN) model for continuous fine-grai...
research
05/07/2020

Hierarchical Attention Network for Action Segmentation

The temporal segmentation of events is an essential task and a precursor...
research
12/18/2018

Multi-Level Sequence GAN for Group Activity Recognition

We propose a novel semi-supervised, Multi-Level Sequential Generative Ad...
research
12/16/2019

Predicting the Future: A Jointly Learnt Model for Action Anticipation

Inspired by human neurological structures for action anticipation, we pr...
research
03/23/2022

How Do You Do It? Fine-Grained Action Understanding with Pseudo-Adverbs

We aim to understand how actions are performed and identify subtle diffe...
research
09/25/2022

Hand Hygiene Assessment via Joint Step Segmentation and Key Action Scorer

Hand hygiene is a standard six-step hand-washing action proposed by the ...
research
08/21/2023

UbiPhysio: Support Daily Functioning, Fitness, and Rehabilitation with Action Understanding and Feedback in Natural Language

We introduce UbiPhysio, a milestone framework that delivers fine-grained...

Please sign up or login with your details

Forgot password? Click here to reset