Cascaded Pyramid Mining Network for Weakly Supervised Temporal Action Localization

by   Haisheng Su, et al.

Weakly supervised temporal action localization, which aims at temporally locating action instances in untrimmed videos using only video-level class labels during training, is an important yet challenging problem in video analysis. Many current methods adopt the "localization by classification" framework: first do video classification, then locate temporal area contributing to the results most. However, this framework fails to locate the entire action instances and gives little consideration to the local context. In this paper, we present a novel architecture called Cascaded Pyramid Mining Network (CPMN) to address these issues using two effective modules. First, to discover the entire temporal interval of specific action, we design a two-stage cascaded module with proposed Online Adversarial Erasing (OAE) mechanism, where new and complementary regions are mined through feeding the erased feature maps of discovered regions back to the system. Second, to exploit hierarchical contextual information in videos and reduce missing detections, we design a pyramid module which produces a scale-invariant attention map through combining the feature maps from different levels. Final, we aggregate the results of two modules to perform action localization via locating high score areas in temporal Class Activation Sequence (CAS). Extensive experiments conducted on THUMOS14 and ActivityNet-1.3 datasets demonstrate the effectiveness of our method.


page 8

page 14


Transferable Knowledge-Based Multi-Granularity Aggregation Network for Temporal Action Localization: Submission to ActivityNet Challenge 2021

This technical report presents an overview of our solution used in the s...

Adversarial Seeded Sequence Growing for Weakly-Supervised Temporal Action Localization

Temporal action localization is an important yet challenging research to...

Weakly-Supervised Temporal Action Localization by Inferring Snippet-Feature Affinity

Weakly-supervised temporal action localization aims to locate action reg...

Weakly-Supervised Action Localization by Hierarchically-structured Latent Attention Modeling

Weakly-supervised action localization aims to recognize and localize act...

Decoupling Localization and Classification in Single Shot Temporal Action Detection

Video temporal action detection aims to temporally localize and recogniz...

Pyramid Region-based Slot Attention Network for Temporal Action Proposal Generation

It has been found that temporal action proposal generation, which aims t...

Video Self-Stitching Graph Network for Temporal Action Localization

Temporal action localization (TAL) in videos is a challenging task, espe...

Please sign up or login with your details

Forgot password? Click here to reset