Few-shot Action Recognition with Implicit Temporal Alignment and Pair Similarity Optimization

10/13/2020
by   Congqi Cao, et al.
8

Few-shot learning aims to recognize instances from novel classes with few labeled samples, which has great value in research and application. Although there has been a lot of work in this area recently, most of the existing work is based on image classification tasks. Video-based few-shot action recognition has not been explored well and remains challenging: 1) the differences of implementation details among different papers make a fair comparison difficult; 2) the wide variations and misalignment of temporal sequences make the video-level similarity comparison difficult; 3) the scarcity of labeled data makes the optimization difficult. To solve these problems, this paper presents 1) a specific setting to evaluate the performance of few-shot action recognition algorithms; 2) an implicit sequence-alignment algorithm for better video-level similarity comparison; 3) an advanced loss for few-shot learning to optimize pair similarity with limited data. Specifically, we propose a novel few-shot action recognition framework that uses long short-term memory following 3D convolutional layers for sequence modeling and alignment. Circle loss is introduced to maximize the within-class similarity and minimize the between-class similarity flexibly towards a more definite convergence target. Instead of using random or ambiguous experimental settings, we set a concrete criterion analogous to the standard image-based few-shot learning setting for few-shot action recognition evaluation. Extensive experiments on two datasets demonstrate the effectiveness of our proposed method.

READ FULL TEXT

page 2

page 7

page 10

research
09/14/2019

Metric-Based Few-Shot Learning for Video Action Recognition

In the few-shot scenario, a learner must effectively generalize to unsee...
research
05/10/2023

Few-shot Action Recognition via Intra- and Inter-Video Information Maximization

Current few-shot action recognition involves two primary sources of info...
research
05/11/2021

Learning Implicit Temporal Alignment for Few-shot Video Classification

Few-shot video classification aims to learn new video categories with on...
research
06/27/2019

Few-Shot Video Classification via Temporal Alignment

There is a growing interest in learning a model which could recognize no...
research
07/10/2021

TTAN: Two-Stage Temporal Alignment Network for Few-shot Action Recognition

Few-shot action recognition aims to recognize novel action classes (quer...
research
04/08/2021

Few-Shot Action Recognition with Compromised Metric via Optimal Transport

Although vital to computer vision systems, few-shot action recognition i...
research
04/30/2019

Curvature: A signature for Action Recognition in Video Sequences

In this paper, a novel signature of human action recognition, namely the...

Please sign up or login with your details

Forgot password? Click here to reset