Generalized Zero-Shot Learning for Action Recognition with Web-Scale Video Data

10/20/2017
by   Kun Liu, et al.
0

Action recognition in surveillance video makes our life safer by detecting the criminal events or predicting violent emergencies. However, efficient action recognition is not free of difficulty. First, there are so many action classes in daily life that we cannot pre-define all possible action classes beforehand. Moreover, it is very hard to collect real-word videos for certain particular actions such as steal and street fight due to legal restrictions and privacy protection. These challenges make existing data-driven recognition methods insufficient to attain desired performance. Zero-shot learning is potential to be applied to solve these issues since it can perform classification without positive example. Nevertheless, current zero-shot learning algorithms have been studied under the unreasonable setting where seen classes are absent during the testing phase. Motivated by this, we study the task of action recognition in surveillance video under a more realistic generalized zero-shot setting, where testing data contains both seen and unseen classes. To our best knowledge, this is the first work to study video action recognition under the generalized zero-shot setting. We firstly perform extensive empirical studies on several existing zero-shot leaning approaches under this new setting on a web-scale video data. Our experimental results demonstrate that, under the generalize setting, typical zero-shot learning methods are no longer effective for the dataset we applied. Then, we propose a method for action recognition by deploying generalized zero-shot learning, which transfers the knowledge of web video to detect the anomalous actions in surveillance videos. To verify the effectiveness of our proposed method, we further construct a new surveillance video dataset consisting of nine action classes related to the public safety situation.

READ FULL TEXT

page 5

page 12

research
01/18/2021

CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognition

Zero-shot action recognition is the task of recognizing action classes w...
research
08/05/2021

Elaborative Rehearsal for Zero-shot Action Recognition

The growing number of action classes has posed a new challenge for video...
research
01/21/2023

Improving Accuracy of Zero-Shot Action Recognition with Handcrafted Features

With the development of machine learning, datasets for models are gettin...
research
09/20/2019

Retro-Actions: Learning 'Close' by Time-Reversing 'Open' Videos

We investigate video transforms that result in class-homogeneous label-t...
research
09/16/2020

Knowledge Guided Learning: Towards Open Domain Egocentric Action Recognition with Zero Supervision

Advances in deep learning have enabled the development of models that ha...
research
09/15/2017

Multi-Label Zero-Shot Human Action Recognition via Joint Latent Embedding

Human action recognition refers to automatic recognizing human actions f...
research
12/08/2019

Zero-shot Recognition of Complex Action Sequences

Zero-shot video classification for fine-grained activity recognition has...

Please sign up or login with your details

Forgot password? Click here to reset