Towards High-Quality Temporal Action Detection with Sparse Proposals

09/18/2021
by   Jiannan Wu, et al.
0

Temporal Action Detection (TAD) is an essential and challenging topic in video understanding, aiming to localize the temporal segments containing human action instances and predict the action categories. The previous works greatly rely upon dense candidates either by designing varying anchors or enumerating all the combinations of boundaries on video sequences; therefore, they are related to complicated pipelines and sensitive hand-crafted designs. Recently, with the resurgence of Transformer, query-based methods have tended to become the rising solutions for their simplicity and flexibility. However, there still exists a performance gap between query-based methods and well-established methods. In this paper, we identify the main challenge lies in the large variants of action duration and the ambiguous boundaries for short action instances; nevertheless, quadratic-computational global attention prevents query-based methods to build multi-scale feature maps. Towards high-quality temporal action detection, we introduce Sparse Proposals to interact with the hierarchical features. In our method, named SP-TAD, each proposal attends to a local segment feature in the temporal feature pyramid. The local interaction enables utilization of high-resolution features to preserve action instances details. Extensive experiments demonstrate the effectiveness of our method, especially under high tIoU thresholds. E.g., we achieve the state-of-the-art performance on THUMOS14 (45.7 mAP@Avg) and competitive results on ActivityNet-1.3 (32.99 will be made available at https://github.com/wjn922/SP-TAD.

READ FULL TEXT
research
06/08/2018

BSN: Boundary Sensitive Network for Temporal Action Proposal Generation

Temporal action proposal generation is an important yet challenging prob...
research
06/18/2021

End-to-end Temporal Action Detection with Transformer

Temporal action detection (TAD) aims to determine the semantic label and...
research
12/07/2021

DCAN: Improving Temporal Action Detection via Dual Context Aggregation

Temporal action detection aims to locate the boundaries of action in the...
research
07/14/2022

Proposal-Free Temporal Action Detection via Global Segmentation Mask Learning

Existing temporal action detection (TAD) methods rely on generating an o...
research
08/25/2022

Adaptive Perception Transformer for Temporal Action Localization

Temporal action localization aims to predict the boundary and category o...
research
11/26/2019

SRG: Snippet Relatedness-based Temporal Action Proposal Generator

Recent temporal action proposal generation approaches have suggested int...
research
03/13/2023

TriDet: Temporal Action Detection with Relative Boundary Modeling

In this paper, we present a one-stage framework TriDet for temporal acti...

Please sign up or login with your details

Forgot password? Click here to reset