CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognition

by   Shreyank N Gowda, et al.

Zero-shot action recognition is the task of recognizing action classes without visual examples, only with a semantic embedding which relates unseen to seen classes. The problem can be seen as learning a function which generalizes well to instances of unseen classes without losing discrimination between classes. Neural networks can model the complex boundaries between visual classes, which explains their success as supervised models. However, in zero-shot learning, these highly specialized class boundaries may not transfer well from seen to unseen classes. In this paper, we propose a clustering-based model, which considers all training samples at once, instead of optimizing for each instance individually. We optimize the clustering using Reinforcement Learning which we show is critical for our approach to work. We call the proposed method CLASTER and observe that it consistently improves over the state-of-the-art in all standard datasets, UCF101, HMDB51, and Olympic Sports; both in the standard zero-shot evaluation and the generalized zero-shot learning.



There are no comments yet.


page 4

page 8


From Classical to Generalized Zero-Shot Learning: a Simple Adaptation Process

Zero-shot learning (ZSL) is concerned with the recognition of previously...

A New Split for Evaluating True Zero-Shot Action Recognition

Zero-shot action recognition is the task of classifying action categorie...

A Unified approach for Conventional Zero-shot, Generalized Zero-shot and Few-shot Learning

Prevalent techniques in zero-shot learning do not generalize well to oth...

Skeleton based Zero Shot Action Recognition in Joint Pose-Language Semantic Space

How does one represent an action? How does one describe an action that w...

Generalized Zero-Shot Learning for Action Recognition with Web-Scale Video Data

Action recognition in surveillance video makes our life safer by detecti...

GAN for Vision, KG for Relation: a Two-stage Deep Network for Zero-shot Action Recognition

Zero-shot action recognition can recognize samples of unseen classes tha...

Improving Generalized Zero-Shot Learning by Semantic Discriminator

It is a recognized fact that the classification accuracy of unseen class...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.