Bag of Visual Words and Fusion Methods for Action Recognition: Comprehensive Study and Good Practice

05/18/2014
by   Xiaojiang Peng, et al.
0

Video based action recognition is one of the important and challenging problems in computer vision research. Bag of Visual Words model (BoVW) with local features has become the most popular method and obtained the state-of-the-art performance on several realistic datasets, such as the HMDB51, UCF50, and UCF101. BoVW is a general pipeline to construct a global representation from a set of local features, which is mainly composed of five steps: (i) feature extraction, (ii) feature pre-processing, (iii) codebook generation, (iv) feature encoding, and (v) pooling and normalization. Many efforts have been made in each step independently in different scenarios and their effect on action recognition is still unknown. Meanwhile, video data exhibits different views of visual pattern, such as static appearance and motion dynamics. Multiple descriptors are usually extracted to represent these different views. Many feature fusion methods have been developed in other areas and their influence on action recognition has never been investigated before. This paper aims to provide a comprehensive study of all steps in BoVW and different fusion methods, and uncover some good practice to produce a state-of-the-art action recognition system. Specifically, we explore two kinds of local features, ten kinds of encoding methods, eight kinds of pooling and normalization strategies, and three kinds of fusion methods. We conclude that every step is crucial for contributing to the final recognition rate. Furthermore, based on our comprehensive study, we propose a simple yet effective representation, called hybrid representation, by exploring the complementarity of different BoVW frameworks and local descriptors. Using this representation, we obtain the state-of-the-art on the three challenging datasets: HMDB51 (61.1

READ FULL TEXT

page 4

page 12

page 17

research
05/29/2014

Feature sampling and partitioning for visual vocabulary generation on large action classification datasets

The recent trend in action recognition is towards larger datasets, an in...
research
01/28/2015

Feature Sampling Strategies for Action Recognition

Although dense local spatial-temporal features with bag-of-features repr...
research
11/25/2020

Recent Progress in Appearance-based Action Recognition

Action recognition, which is formulated as a task to identify various hu...
research
03/23/2017

A Bag-of-Words Equivalent Recurrent Neural Network for Action Recognition

The traditional bag-of-words approach has found a wide range of applicat...
research
04/21/2015

A robust and efficient video representation for action recognition

This paper introduces a state-of-the-art video representation and applie...
research
12/14/2016

Scale Coding Bag of Deep Features for Human Attribute and Action Recognition

Most approaches to human attribute and action recognition in still image...
research
02/21/2017

Scene Recognition by Combining Local and Global Image Descriptors

Object recognition is an important problem in computer vision, having di...

Please sign up or login with your details

Forgot password? Click here to reset