Weakly Supervised Action Segmentation Using Mutual Consistency

04/05/2019
by   Yaser Souri, et al.
26

Action segmentation is the task of predicting the actions in each frame of a video. Because of the high cost of preparing training videos with full supervision for action segmentation, weakly supervised approaches which are able to learn only from transcripts are very appealing. In this paper, we propose a new approach for weakly supervised action segmentation based on a two branch network. The two branches of our network predict two redundant but different representations for action segmentation. During training we introduce a new mutual consistency loss (MuCon) that enforces that these two representations are consistent. Using MuCon and a transcript prediction loss, our network achieves state-of-the-art results for action segmentation and action alignment while being fully differentiable and faster to train since it does not require a costly alignment step during training.

READ FULL TEXT
research
03/31/2020

SCT: Set Constrained Temporal Transformer for Set Supervised Action Segmentation

Temporal action segmentation is a topic of increasing interest, however,...
research
05/19/2020

On Evaluating Weakly Supervised Action Segmentation Methods

Action segmentation is the task of temporally segmenting every frame of ...
research
08/09/2021

FIFA: Fast Inference Approximation for Action Segmentation

We introduce FIFA, a fast approximate inference method for action segmen...
research
03/11/2021

Temporal Action Segmentation from Timestamp Supervision

Temporal action segmentation approaches have been very successful recent...
research
01/09/2019

D^3TW: Discriminative Differentiable Dynamic Time Warping for Weakly Supervised Action Alignment and Segmentation

We address weakly-supervised action alignment and segmentation in videos...
research
03/29/2020

Learning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection

We address weakly-supervised video actor-action segmentation (VAAS), whi...
research
03/24/2022

Weakly-Supervised Online Action Segmentation in Multi-View Instructional Videos

This paper addresses a new problem of weakly-supervised online action se...

Please sign up or login with your details

Forgot password? Click here to reset