DeepAI AI Chat
Log In Sign Up

G-TAD: Sub-Graph Localization for Temporal Action Detection

by   Mengmeng Xu, et al.
King Abdullah University of Science and Technology

Temporal action detection is a fundamental yet challenging task in video understanding. Video context is a critical cue to effectively detect actions, but current works mainly focus on temporal context, while neglecting semantic con-text as well as other important context properties. In this work, we propose a graph convolutional network (GCN) model to adaptively incorporate multi-level semantic context into video features and cast temporal action detection as a sub-graph localization problem. Specifically, we formulate video snippets as graph nodes, snippet-snippet cor-relations as edges, and actions associated with context as target sub-graphs. With graph convolution as the basic operation, we design a GCN block called GCNeXt, which learns the features of each node by aggregating its context and dynamically updates the edges in the graph. To localize each sub-graph, we also design a SGAlign layer to embed each sub-graph into the Euclidean space. Extensive experiments show that G-TAD is capable of finding effective video context without extra supervision and achieves state-of-the-art performance on two detection benchmarks. On ActityNet-1.3, we obtain an average mAP of 34.09 40.16


page 3

page 8


Augmented Transformer with Adaptive Graph for Temporal Action Proposal Generation

Temporal action proposal generation (TAPG) is a fundamental and challeng...

Video Self-Stitching Graph Network for Temporal Action Localization

Temporal action localization (TAL) in videos is a challenging task, espe...

PcmNet: Position-Sensitive Context Modeling Network for Temporal Action Localization

Temporal action localization is an important and challenging task that a...

Temporal Extension Module for Skeleton-Based Action Recognition

We present a module that extends the temporal graph of a graph convoluti...

Representing Videos as Discriminative Sub-graphs for Action Recognition

Human actions are typically of combinatorial structures or patterns, i.e...

Multi-Level Graph Convolutional Network with Automatic Graph Learning for Hyperspectral Image Classification

Nowadays, deep learning methods, especially the Graph Convolutional Netw...

What's Wrong with the Bottom-up Methods in Arbitrary-shape Scene Text Detection

The latest trend in the bottom-up perspective for arbitrary-shape scene ...