Adversarial Seeded Sequence Growing for Weakly-Supervised Temporal Action Localization

by   Chengwei Zhang, et al.

Temporal action localization is an important yet challenging research topic due to its various applications. Since the frame-level or segment-level annotations of untrimmed videos require amounts of labor expenditure, studies on the weakly-supervised action detection have been springing up. However, most of existing frameworks rely on Class Activation Sequence (CAS) to localize actions by minimizing the video-level classification loss, which exploits the most discriminative parts of actions but ignores the minor regions. In this paper, we propose a novel weakly-supervised framework by adversarial learning of two modules for eliminating such demerits. Specifically, the first module is designed as a well-designed Seeded Sequence Growing (SSG) Network for progressively extending seed regions (namely the highly reliable regions initialized by a CAS-based framework) to their expected boundaries. The second module is a specific classifier for mining trivial or incomplete action regions, which is trained on the shared features after erasing the seeded regions activated by SSG. In this way, a whole network composed of these two modules can be trained in an adversarial manner. The goal of the adversary is to mine features that are difficult for the action classifier. That is, erasion from SSG will force the classifier to discover minor or even new action regions on the input feature sequence, and the classifier will drive the seeds to grow, alternately. At last, we could obtain the action locations and categories from the well-trained SSG and the classifier. Extensive experiments on two public benchmarks THUMOS'14 and ActivityNet1.3 demonstrate the impressive performance of our proposed method compared with the state-of-the-arts.


page 1

page 4

page 7


AutoLoc: Weakly-supervised Temporal Action Localization

Temporal Action Localization (TAL) in untrimmed video is important for m...

Cascaded Pyramid Mining Network for Weakly Supervised Temporal Action Localization

Weakly supervised temporal action localization, which aims at temporally...

CoLA: Weakly-Supervised Temporal Action Localization with Snippet Contrastive Learning

Weakly-supervised temporal action localization (WS-TAL) aims to localize...

Marginalized Average Attentional Network for Weakly-Supervised Learning

In weakly-supervised temporal action localization, previous works have f...

Equivalent Classification Mapping for Weakly Supervised Temporal Action Localization

Weakly supervised temporal action localization is a newly emerging yet w...

Adversarial Complementary Learning for Weakly Supervised Object Localization

In this work, we propose Adversarial Complementary Learning (ACoL) to au...

Adaptive Mutual Supervision for Weakly-Supervised Temporal Action Localization

Weakly-supervised temporal action localization aims to localize actions ...

Please sign up or login with your details

Forgot password? Click here to reset