Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentation

03/20/2021
by   M. Saquib Sarfraz, et al.
1

Action segmentation refers to inferring boundaries of semantically consistent visual concepts in videos and is an important requirement for many video understanding tasks. For this and other video understanding tasks, supervised approaches have achieved encouraging performance but require a high volume of detailed frame-level annotations. We present a fully automatic and unsupervised approach for segmenting actions in a video that does not require any training. Our proposal is an effective temporally-weighted hierarchical clustering algorithm that can group semantically consistent frames of the video. Our main finding is that representing a video with a 1-nearest neighbor graph by taking into account the time progression is sufficient to form semantically and temporally consistent clusters of frames where each cluster may represent some action in the video. Additionally, we establish strong unsupervised baselines for action segmentation and show significant performance improvements over published unsupervised methods on five challenging action segmentation datasets. Our code is available at https://github.com/ssarfraz/FINCH-Clustering/tree/master/TW-FINCH

READ FULL TEXT

page 2

page 8

research
09/28/2021

VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding

We present VideoCLIP, a contrastive approach to pre-train a unified mode...
research
11/12/2021

Temporally-Consistent Surface Reconstruction using Metrically-Consistent Atlases

We propose a method for unsupervised reconstruction of a temporally-cons...
research
03/09/2023

TAEC: Unsupervised Action Segmentation with Temporal-Aware Embedding and Clustering

Temporal action segmentation in untrimmed videos has gained increased at...
research
11/22/2021

Towards Tokenized Human Dynamics Representation

For human action understanding, a popular research direction is to analy...
research
10/29/2022

Unsupervised Audio-Visual Lecture Segmentation

Over the last decade, online lecture videos have become increasingly pop...
research
04/21/2019

Automatic Temporally Coherent Video Colorization

Greyscale image colorization for applications in image restoration has s...
research
12/18/2014

Unsupervised Learning of Spatiotemporally Coherent Metrics

Current state-of-the-art classification and detection algorithms rely on...

Please sign up or login with your details

Forgot password? Click here to reset