Locally-Consistent Deformable Convolution Networks for Fine-Grained Action Detection

11/21/2018
by   Khoi-Nguyen C. Mac, et al.
0

Fine-grained action detection is an important task with numerous applications in robotics, human-computer interaction, and video surveillance. Several existing methods use the popular two-stream approach, which learns the spatial and temporal information independently from one another. Additionally, the temporal stream of the model usually relies on extracted optical flow from the video stream. In this work, we propose a deep learning model to jointly learn both spatial and temporal information without the necessity of optical flow. We also propose a novel convolution, namely locally-consistent deformable convolution, which enforces a local coherency constraint on the receptive fields. The model produces short-term spatio-temporal features, which can be flexibly used in conjunction with other long-temporal modeling networks. The proposed features used in conjunction with a major state-of-the-art long-temporal model ED-TCN outperforms the original ED-TCN implementation on two fine-grained action datasets: 50 Salads and GTEA, by up to 10.0 and also outperforms the recent state-of-the-art TDRN, by up to 5.9

READ FULL TEXT

page 2

page 8

research
08/22/2019

Multi-Stream Single Shot Spatial-Temporal Action Detection

We present a 3D Convolutional Neural Networks (CNNs) based single shot d...
research
08/01/2016

Exploiting Temporal Information for DCNN-based Fine-Grained Object Classification

Fine-grained classification is a relatively new field that has concentra...
research
10/13/2022

Real-time Action Recognition for Fine-Grained Actions and The Hand Wash Dataset

In this paper we present a three-stream algorithm for real-time action r...
research
01/17/2020

Temporal Interlacing Network

For a long time, the vision community tries to learn the spatio-temporal...
research
11/08/2022

Two-stream Multi-dimensional Convolutional Network for Real-time Violence Detection

The increasing number of surveillance cameras and security concerns have...
research
08/18/2016

Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction

Predicting an interaction before it is fully executed is very important ...

Please sign up or login with your details

Forgot password? Click here to reset