DeepAI AI Chat
Log In Sign Up

Learning Implicit Temporal Alignment for Few-shot Video Classification

by   Songyang Zhang, et al.

Few-shot video classification aims to learn new video categories with only a few labeled examples, alleviating the burden of costly annotation in real-world applications. However, it is particularly challenging to learn a class-invariant spatial-temporal representation in such a setting. To address this, we propose a novel matching-based few-shot learning strategy for video sequences in this work. Our main idea is to introduce an implicit temporal alignment for a video pair, capable of estimating the similarity between them in an accurate and robust manner. Moreover, we design an effective context encoding module to incorporate spatial and feature channel context, resulting in better modeling of intra-class variations. To train our model, we develop a multi-task loss for learning video matching, leading to video features with better generalization. Extensive experimental results on two challenging benchmarks, show that our method outperforms the prior arts with a sizable margin on SomethingSomething-V2 and competitive results on Kinetics.


Temporal Alignment Prediction for Few-Shot Video Classification

The goal of few-shot video classification is to learn a classification m...

Few-Shot Video Classification via Temporal Alignment

There is a growing interest in learning a model which could recognize no...

Few-shot Action Recognition with Implicit Temporal Alignment and Pair Similarity Optimization

Few-shot learning aims to recognize instances from novel classes with fe...

Part-aware Prototype Network for Few-shot Semantic Segmentation

Few-shot semantic segmentation aims to learn to segment new object class...

Few-shot Action Recognition via Intra- and Inter-Video Information Maximization

Current few-shot action recognition involves two primary sources of info...

Cross-Class Relevance Learning for Temporal Concept Localization

We present a novel Cross-Class Relevance Learning approach for the task ...

DeepEMD: Few-Shot Image Classification with Differentiable Earth Mover's Distance and Structured Classifiers

In this paper, we address the few-shot classification task from a new pe...