TTAN: Two-Stage Temporal Alignment Network for Few-shot Action Recognition

07/10/2021
by   Shuyuan Li, et al.
3

Few-shot action recognition aims to recognize novel action classes (query) using just a few samples (support). The majority of current approaches follow the metric learning paradigm, which learns to compare the similarity between videos. Recently, it has been observed that directly measuring this similarity is not ideal since different action instances may show distinctive temporal distribution, resulting in severe misalignment issues across query and support videos. In this paper, we arrest this problem from two distinct aspects – action duration misalignment and motion evolution misalignment. We address them sequentially through a Two-stage Temporal Alignment Network (TTAN). The first stage performs temporal transformation with the predicted affine warp parameters, while the second stage utilizes a cross-attention mechanism to coordinate the features of the support and query to a consistent evolution. Besides, we devise a novel multi-shot fusion strategy, which takes the misalignment among support samples into consideration. Ablation studies and visualizations demonstrate the role played by both stages in addressing the misalignment. Extensive experiments on benchmark datasets show the potential of the proposed method in achieving state-of-the-art performance for few-shot action recognition.

READ FULL TEXT

page 1

page 4

page 7

research
01/15/2021

Temporal-Relational CrossTransformers for Few-Shot Action Recognition

We propose a novel approach to few-shot action recognition, finding temp...
research
07/12/2022

Compound Prototype Matching for Few-shot Action Recognition

Few-shot action recognition aims to recognize novel action classes using...
research
01/20/2021

Few-shot Action Recognition with Prototype-centered Attentive Learning

Few-shot action recognition aims to recognize action classes with few tr...
research
04/21/2020

TAEN: Temporal Aware Embedding Network for Few-Shot Action Recognition

Classification of a new class entities requires collecting and annotatin...
research
08/14/2023

On the Importance of Spatial Relations for Few-shot Action Recognition

Deep learning has achieved great success in video recognition, yet still...
research
10/13/2020

Few-shot Action Recognition with Implicit Temporal Alignment and Pair Similarity Optimization

Few-shot learning aims to recognize instances from novel classes with fe...
research
07/05/2023

Task-Specific Alignment and Multiple Level Transformer for Few-Shot Action Recognition

In the research field of few-shot learning, the main difference between ...

Please sign up or login with your details

Forgot password? Click here to reset