Action Shuffle Alternating Learning for Unsupervised Action Segmentation

04/05/2021
by   Jun Li, et al.
0

This paper addresses unsupervised action segmentation. Prior work captures the frame-level temporal structure of videos by a feature embedding that encodes time locations of frames in the video. We advance prior work with a new self-supervised learning (SSL) of a feature embedding that accounts for both frame- and action-level structure of videos. Our SSL trains an RNN to recognize positive and negative action sequences, and the RNN's hidden layer is taken as our new action-level feature embedding. The positive and negative sequences consist of action segments sampled from videos, where in the former the sampled action segments respect their time ordering in the video, and in the latter they are shuffled. As supervision of actions is not available and our SSL requires access to action segments, we specify an HMM that explicitly models action lengths, and infer a MAP action segmentation with the Viterbi algorithm. The resulting action segmentation is used as pseudo-ground truth for estimating our action-level feature embedding and updating the HMM. We alternate the above steps within the Generalized EM framework, which ensures convergence. Our evaluation on the Breakfast, YouTube Instructions, and 50Salads datasets gives superior results to those of the state of the art.

READ FULL TEXT
research
09/28/2019

Weakly Supervised Energy-Based Learning for Action Segmentation

This paper is about labeling video frames with action classes under weak...
research
02/27/2020

Set-Constrained Viterbi for Set-Supervised Action Segmentation

This paper is about weakly supervised action segmentation, where ground ...
research
03/09/2023

TAEC: Unsupervised Action Segmentation with Temporal-Aware Embedding and Clustering

Temporal action segmentation in untrimmed videos has gained increased at...
research
04/05/2021

Anchor-Constrained Viterbi for Set-Supervised Action Segmentation

This paper is about action segmentation under weak supervision in traini...
research
09/12/2023

OTAS: Unsupervised Boundary Detection for Object-Centric Temporal Action Segmentation

Temporal action segmentation is typically achieved by discovering the dr...
research
05/31/2020

Motion2Vec: Semi-Supervised Representation Learning from Surgical Videos

Learning meaningful visual representations in an embedding space can fac...
research
09/30/2017

Unsupervised Segmentation of Action Segments in Egocentric Videos using Gaze

Unsupervised segmentation of action segments in egocentric videos is a d...

Please sign up or login with your details

Forgot password? Click here to reset