Weakly Supervised Temporal Action Localization Through Learning Explicit Subspaces for Action and Context

03/30/2021
by   Ziyi Liu, et al.
0

Weakly-supervised Temporal Action Localization (WS-TAL) methods learn to localize temporal starts and ends of action instances in a video under only video-level supervision. Existing WS-TAL methods rely on deep features learned for action recognition. However, due to the mismatch between classification and localization, these features cannot distinguish the frequently co-occurring contextual background, i.e., the context, and the actual action instances. We term this challenge action-context confusion, and it will adversely affect the action localization accuracy. To address this challenge, we introduce a framework that learns two feature subspaces respectively for actions and their context. By explicitly accounting for action visual elements, the action instances can be localized more precisely without the distraction from the context. To facilitate the learning of these two feature subspaces with only video-level categorical labels, we leverage the predictions from both spatial and temporal streams for snippets grouping. In addition, an unsupervised learning task is introduced to make the proposed module focus on mining temporal information. The proposed approach outperforms state-of-the-art WS-TAL methods on three benchmarks, i.e., THUMOS14, ActivityNet v1.2 and v1.3 datasets.

READ FULL TEXT

page 2

page 6

research
03/28/2021

ACSNet: Action-Context Separation Network for Weakly Supervised Temporal Action Localization

The object of Weakly-supervised Temporal Action Localization (WS-TAL) is...
research
06/23/2022

Learning to Refactor Action and Co-occurrence Features for Temporal Action Localization

The main challenge of Temporal Action Localization is to retrieve subtle...
research
03/30/2023

JCDNet: Joint of Common and Definite phases Network for Weakly Supervised Temporal Action Localization

Weakly-supervised temporal action localization aims to localize action i...
research
04/25/2023

Weakly-Supervised Temporal Action Localization with Bidirectional Semantic Consistency Constraint

Weakly Supervised Temporal Action Localization (WTAL) aims to classify a...
research
04/10/2019

Attentive Action and Context Factorization

We propose a method for human action recognition, one that can localize ...
research
08/18/2020

Equivalent Classification Mapping for Weakly Supervised Temporal Action Localization

Weakly supervised temporal action localization is a newly emerging yet w...
research
03/10/2022

OpenTAL: Towards Open Set Temporal Action Localization

Temporal Action Localization (TAL) has experienced remarkable success un...

Please sign up or login with your details

Forgot password? Click here to reset