CTRN: Class-Temporal Relational Network for Action Detection

10/26/2021
by   Rui Dai, et al.
0

Action detection is an essential and challenging task, especially for densely labelled datasets of untrimmed videos. There are many real-world challenges in those datasets, such as composite action, co-occurring action, and high temporal variation of instance duration. For handling these challenges, we propose to explore both the class and temporal relations of detected actions. In this work, we introduce an end-to-end network: Class-Temporal Relational Network (CTRN). It contains three key components: (1) The Representation Transform Module filters the class-specific features from the mixed representations to build graph-structured data. (2) The Class-Temporal Module models the class and temporal relations in a sequential manner. (3) G-classifier leverages the privileged knowledge of the snippet-wise co-occurring action pairs to further improve the co-occurring action detection. We evaluate CTRN on three challenging densely labelled datasets and achieve state-of-the-art performance, reflecting the effectiveness and robustness of our method.

READ FULL TEXT

page 2

page 8

research
12/07/2021

MS-TCT: Multi-Scale Temporal ConvTransformer for Action Detection

Action detection is an essential and challenging task, especially for de...
research
03/17/2020

A Novel Online Action Detection Framework from Untrimmed Video Streams

Online temporal action localization from an untrimmed video stream is a ...
research
11/22/2015

End-to-end Learning of Action Detection from Frame Glimpses in Videos

In this work we introduce a fully end-to-end approach for action detecti...
research
07/18/2022

Leveraging Action Affinity and Continuity for Semi-supervised Temporal Action Segmentation

We present a semi-supervised learning approach to the temporal action se...
research
04/08/2019

Relational Action Forecasting

This paper focuses on multi-person action forecasting in videos. More pr...
research
05/02/2017

Cascaded Boundary Regression for Temporal Action Detection

Temporal action detection in long videos is an important problem. State-...
research
07/20/2022

ERA: Expert Retrieval and Assembly for Early Action Prediction

Early action prediction aims to successfully predict the class label of ...

Please sign up or login with your details

Forgot password? Click here to reset