Zero-Shot Action Recognition with Transformer-based Video Semantic Embedding

03/10/2022
by   Keval Doshi, et al.
0

While video action recognition has been an active area of research for several years, zero-shot action recognition has only recently started gaining traction. However, there is a lack of a formal definition for the zero-shot learning paradigm leading to uncertainty about classes that can be considered as previously unseen. In this work, we take a new comprehensive look at the inductive zero-shot action recognition problem from a realistic standpoint. Specifically, we advocate for a concrete formulation for zero-shot action recognition that avoids an exact overlap between the training and testing classes and also limits the intra-class variance; and propose a novel end-to-end trained transformer model which is capable of capturing long range spatiotemporal dependencies efficiently, contrary to existing approaches which use 3D-CNNs. The proposed approach outperforms the existing state-of-the-art algorithms in many settings on all benchmark datasets by a wide margin.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/27/2021

A New Split for Evaluating True Zero-Shot Action Recognition

Zero-shot action recognition is the task of classifying action categorie...
research
07/21/2019

TARN: Temporal Attentive Relation Network for Few-Shot and Zero-Shot Action Recognition

In this paper we propose a novel Temporal Attentive Relation Network (TA...
research
08/09/2023

Seeing in Flowing: Adapting CLIP for Action Recognition with Motion Prompts Learning

The Contrastive Language-Image Pre-training (CLIP) has recently shown re...
research
10/13/2017

Recent Advances in Zero-shot Recognition

With the recent renaissance of deep convolution neural networks, encoura...
research
03/03/2020

Rethinking Zero-shot Video Classification: End-to-end Training for Realistic Applications

Trained on large datasets, deep learning (DL) can accurately classify vi...
research
01/21/2023

Improving Accuracy of Zero-Shot Action Recognition with Handcrafted Features

With the development of machine learning, datasets for models are gettin...
research
11/26/2016

Multi-Task Zero-Shot Action Recognition with Prioritised Data Augmentation

Zero-Shot Learning (ZSL) promises to scale visual recognition by bypassi...

Please sign up or login with your details

Forgot password? Click here to reset