Unsupervised learning of action classes with continuous temporal embedding

04/08/2019
by   Anna Kukleva, et al.
ibm
University of Bonn
0

The task of temporally detecting and segmenting actions in untrimmed videos has seen an increased attention recently. One problem in this context arises from the need to define and label action boundaries to create annotations for training which is very time and cost intensive. To address this issue, we propose an unsupervised approach for learning action classes from untrimmed video sequences. To this end, we use a continuous temporal embedding of framewise features to benefit from the sequential nature of activities. Based on the latent space created by the embedding, we identify clusters of temporal segments across all videos that correspond to semantic meaningful action classes. The approach is evaluated on three challenging datasets, namely the Breakfast dataset, YouTube Instructions, and the 50Salads dataset. While previous works assumed that the videos contain the same high level activity, we furthermore show that the proposed approach can also be applied to a more general setting where the content of the videos is unknown.

READ FULL TEXT

page 6

page 7

03/09/2023

TAEC: Unsupervised Action Segmentation with Temporal-Aware Embedding and Clustering

Temporal action segmentation in untrimmed videos has gained increased at...
01/29/2020

Joint Visual-Temporal Embedding for Unsupervised Learning of Actions in Untrimmed Sequences

Understanding the structure of complex activities in videos is one of th...
04/30/2021

Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activities

Action recognition and detection in the context of long untrimmed video ...
09/30/2017

Unsupervised Segmentation of Action Segments in Egocentric Videos using Gaze

Unsupervised segmentation of action segments in egocentric videos is a d...
04/02/2023

From Isolated Islands to Pangea: Unifying Semantic Space for Human Action Understanding

Action understanding matters and attracts attention. It can be formed as...
11/14/2020

TenFor: A Tensor-Based Tool to Extract Interesting Events from Security Forums

How can we get a security forum to "tell" us its activities and events o...
10/11/2018

Globally Continuous and Non-Markovian Activity Analysis from Videos

Automatically recognizing activities in video is a classic problem in vi...

Code Repositories

unsup_temp_embed

Official implementation of the paper: Unsupervised learning of action classes with continuous temporal embedding


view repo

Please sign up or login with your details

Forgot password? Click here to reset