Online Real-time Multiple Spatiotemporal Action Localisation and Prediction

11/25/2016
by   Gurkirt Singh, et al.
0

We present a deep-learning framework for real-time multiple spatio-temporal (S/T) action localisation, classification and early prediction. Current state-of-the-art approaches work offline and are too slow to be useful in real- world settings. To overcome their limitations we introduce two major developments. Firstly, we adopt real-time SSD (Single Shot MultiBox Detector) convolutional neural networks to regress and classify detection boxes in each video frame potentially containing an action of interest. Secondly, we design an original and efficient online algorithm to incrementally construct and label `action tubes' from the SSD frame level detections. As a result, our system is not only capable of performing S/T detection in real time, but can also perform early action prediction in an online fashion. We achieve new state-of-the-art results in both S/T action localisation and early action prediction on the challenging UCF101-24 and J-HMDB-21 benchmarks, even when compared to the top offline competitors. To the best of our knowledge, ours is the first real-time (up to 40fps) system able to perform online S/T action localisation and early action prediction on the untrimmed videos of UCF101-24.

READ FULL TEXT

page 1

page 3

research
04/05/2017

Incremental Tube Construction for Human Action Detection

Current state-of-the-art action detection systems are tailored for offli...
research
08/31/2020

Online Spatiotemporal Action Detection and Prediction via Causal Representations

In this thesis, we focus on video action understanding problems from an ...
research
10/28/2016

Real-time Online Action Detection Forests using Spatio-temporal Contexts

Online action detection (OAD) is challenging since 1) robust yet computa...
research
11/06/2019

Towards the Use of Neural Networks for Influenza Prediction at Multiple Spatial Resolutions

We introduce the use of a Gated Recurrent Unit (GRU) for influenza predi...
research
02/14/2023

YOWOv2: A Stronger yet Efficient Multi-level Detection Framework for Real-time Spatio-temporal Action Detection

Designing a real-time framework for the spatio-temporal action detection...
research
08/04/2016

Deep Learning for Detecting Multiple Space-Time Action Tubes in Videos

In this work, we propose an approach to the spatiotemporal localisation ...
research
03/15/2018

Temporal Human Action Segmentation via Dynamic Clustering

We present an effective dynamic clustering algorithm for the task of tem...

Please sign up or login with your details

Forgot password? Click here to reset