Joint Learning On The Hierarchy Representation for Fine-Grained Human Action Recognition

by   Mei Chee Leong, et al.
Nanyang Technological University
Agency for Science, Technology and Research

Fine-grained human action recognition is a core research topic in computer vision. Inspired by the recently proposed hierarchy representation of fine-grained actions in FineGym and SlowFast network for action recognition, we propose a novel multi-task network which exploits the FineGym hierarchy representation to achieve effective joint learning and prediction for fine-grained human action recognition. The multi-task network consists of three pathways of SlowOnly networks with gradually increased frame rates for events, sets and elements of fine-grained actions, followed by our proposed integration layers for joint learning and prediction. It is a two-stage approach, where it first learns deep feature representation at each hierarchical level, and is followed by feature encoding and fusion for multi-task learning. Our empirical results on the FineGym dataset achieve a new state-of-the-art performance, with 91.80 3.40


page 1

page 2

page 3

page 4


ConvGRU in Fine-grained Pitching Action Recognition for Action Outcome Prediction

Prediction of the action outcome is a new challenge for a robot collabor...

Combined CNN Transformer Encoder for Enhanced Fine-grained Human Action Recognition

Fine-grained action recognition is a challenging task in computer vision...

Action Recognition with Coarse-to-Fine Deep Feature Integration and Asynchronous Fusion

Action recognition is an important yet challenging task in computer visi...

Rugby-Bot: Utilizing Multi-Task Learning Fine-Grained Features for Rugby League Analysis

Sporting events are extremely complex and require a multitude of metrics...

An Analysis of Action Recognition Datasets for Language and Vision Tasks

A large amount of recent research has focused on tasks that combine lang...

Multi-Expert Human Action Recognition with Hierarchical Super-Class Learning

In still image human action recognition, existing studies have mainly le...

CPR-Coach: Recognizing Composite Error Actions based on Single-class Training

The fine-grained medical action analysis task has received considerable ...

Please sign up or login with your details

Forgot password? Click here to reset