Adaptive Mutual Supervision for Weakly-Supervised Temporal Action Localization

04/06/2021
by   Chen Ju, et al.
0

Weakly-supervised temporal action localization aims to localize actions in untrimmed videos with only video-level action category labels. Most of previous methods ignore the incompleteness issue of Class Activation Sequences (CAS), suffering from trivial localization results. To solve this issue, we introduce an adaptive mutual supervision framework (AMS) with two branches, where the base branch adopts CAS to localize the most discriminative action regions, while the supplementary branch localizes the less discriminative action regions through a novel adaptive sampler. The adaptive sampler dynamically updates the input of the supplementary branch with a sampling weight sequence negatively correlated with the CAS from the base branch, thereby prompting the supplementary branch to localize the action regions underestimated by the base branch. To promote mutual enhancement between these two branches, we construct mutual location supervision. Each branch leverages location pseudo-labels generated from the other branch as localization supervision. By alternately optimizing the two branches in multiple iterations, we progressively complete action regions. Extensive experiments on THUMOS14 and ActivityNet1.2 demonstrate that the proposed AMS method significantly outperforms the state-of-the-art methods.

READ FULL TEXT

page 1

page 11

03/28/2021

ACSNet: Action-Context Separation Network for Weakly Supervised Temporal Action Localization

The object of Weakly-supervised Temporal Action Localization (WS-TAL) is...
04/07/2021

ACM-Net: Action Context Modeling Network for Weakly-Supervised Temporal Action Localization

Weakly-supervised temporal action localization aims to localize action i...
08/14/2021

Foreground-Action Consistency Network for Weakly Supervised Temporal Action Localization

As a challenging task of high-level video understanding, weakly supervis...
08/07/2022

Weakly Supervised Online Action Detection for Infant General Movements

To make the earlier medical intervention of infants' cerebral palsy (CP)...
05/10/2021

Action Shuffling for Weakly Supervised Temporal Localization

Weakly supervised action localization is a challenging task with extensi...
08/07/2019

Adversarial Seeded Sequence Growing for Weakly-Supervised Temporal Action Localization

Temporal action localization is an important yet challenging research to...
04/16/2019

Decoupling Localization and Classification in Single Shot Temporal Action Detection

Video temporal action detection aims to temporally localize and recogniz...