DeepAI AI Chat
Log In Sign Up

The Blessings of Unlabeled Background in Untrimmed Videos

by   Yuan Liu, et al.
Alibaba Group
The University of Hong Kong
Nanyang Technological University

Weakly-supervised Temporal Action Localization (WTAL) aims to detect the intervals of action instances with only video-level action labels available during training. The key challenge is how to distinguish the segments of interest from the background segments, which are unlabelled even on the video-level. While previous works treat the background as "curses", we consider it as "blessings". Specifically, we first use causal analysis to point out that the common localization errors are due to the unobserved and un-enumerated confounder that resides ubiquitously in visual recognition. Then, we propose a Temporal Smoothing PCA-based (TS-PCA) deconfounder, which exploits the unlabelled background to model an observed substitute for the confounder, to remove the confounding effect. Note that the proposed deconfounder is model-agnostic and non-intrusive, and hence can be applied in any WTAL method without modification. Through extensive experiments on four state-of-the-art WTAL methods, we show that the deconfounder can improve all of them on the public datasets: THUMOS-14 and ActivityNet-1.3.


page 1

page 8


Background Modeling via Uncertainty Estimation for Weakly-supervised Action Localization

Weakly-supervised temporal action localization aims to detect intervals ...

Forcing the Whole Video as Background: An Adversarial Learning Strategy for Weakly Temporal Action Localization

With video-level labels, weakly supervised temporal action localization ...

Weakly-Supervised Temporal Action Localization Through Local-Global Background Modeling

Weakly-Supervised Temporal Action Localization (WS-TAL) task aims to rec...

Weakly Supervised Temporal Action Localization Through Learning Explicit Subspaces for Action and Context

Weakly-supervised Temporal Action Localization (WS-TAL) methods learn to...

LPAT: Learning to Predict Adaptive Threshold for Weakly-supervised Temporal Action Localization

Recently, Weakly-supervised Temporal Action Localization (WTAL) has been...

Weakly-supervised Action Localization via Hierarchical Mining

Weakly-supervised action localization aims to localize and classify acti...

Slow Motion Matters: A Slow Motion Enhanced Network for Weakly Supervised Temporal Action Localization

Weakly supervised temporal action localization (WTAL) aims to localize a...