CTAP: Complementary Temporal Action Proposal Generation

07/12/2018
by   Jiyang Gao, et al.
0

Temporal action proposal generation is an important task, akin to object proposals, temporal action proposals are intended to capture "clips" or temporal intervals in videos that are likely to contain an action. Previous methods can be divided to two groups: sliding window ranking and actionness score grouping. Sliding windows uniformly cover all segments in videos, but the temporal boundaries are imprecise; grouping based method may have more precise boundaries but it may omit some proposals when the quality of actionness score is low. Based on the complementary characteristics of these two methods, we propose a novel Complementary Temporal Action Proposal (CTAP) generator. Specifically, we apply a Proposal-level Actionness Trustworthiness Estimator (PATE) on the sliding windows proposals to generate the probabilities indicating whether the actions can be correctly detected by actionness scores, the windows with high scores are collected. The collected sliding windows and actionness proposals are then processed by a temporal convolutional neural network for proposal ranking and boundary adjustment. CTAP outperforms state-of-the-art methods on average recall (AR) by a large margin on THUMOS-14 and ActivityNet 1.3 datasets. We further apply CTAP as a proposal generation method in an existing action detector, and show consistent significant improvements.

READ FULL TEXT
research
11/26/2019

SRG: Snippet Relatedness-based Temporal Action Proposal Generator

Recent temporal action proposal generation approaches have suggested int...
research
03/17/2017

TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals

Temporal Action Proposal (TAP) generation is an important problem, as fa...
research
05/02/2017

Cascaded Boundary Regression for Temporal Action Detection

Temporal action detection in long videos is an important problem. State-...
research
09/17/2019

Deep Point-wise Prediction for Action Temporal Proposal

Detecting actions in videos is an important yet challenging task. Previo...
research
08/08/2017

Temporal Context Network for Activity Localization in Videos

We present a Temporal Context Network (TCN) for precise temporal localiz...
research
09/15/2020

BSN++: Complementary Boundary Regressor with Scale-Balanced Relation Modeling for Temporal Action Proposal Generation

Generating human action proposals in untrimmed videos is an important ye...
research
11/21/2018

MAC: Mining Activity Concepts for Language-based Temporal Localization

We address the problem of language-based temporal localization in untrim...

Please sign up or login with your details

Forgot password? Click here to reset