A Real-time Action Prediction Framework by Encoding Temporal Evolution

by   Fahimeh Rezazadegan, et al.

Anticipating future actions is a key component of intelligence, specifically when it applies to real-time systems, such as robots or autonomous cars. While recent work has addressed prediction of raw RGB pixel values in future video frames, we focus on predicting further in future by predicting a summary of moving pixels through a sequence of frames which we call dynamic images. More precisely, given a dynamic image, we predict the motion evolution through next unseen video frames. Since this representation consists of a sequence of frames, we can go one second further into the future compared to the previous work in this field. We employed convolutional LSTMs to train our network on the dynamic images in an unsupervised learning process. Since our final goal is predicting the next action of a complex task such as an assembly task, we exploited labelled actions for the recognition process on top of predicted dynamic images. We show the effectiveness of our method on predicting the next human action in the above-mentioned task through the two-step process of predicting the next dynamic image and recognizing the action which it represents.



There are no comments yet.


page 3

page 4

page 5

page 8


Predicting Deeper into the Future of Semantic Segmentation

The ability to predict and therefore to anticipate the future is an impo...

Deep Sequence Learning for Video Anticipation: From Discrete and Deterministic to Continuous and Stochastic

Video anticipation is the task of predicting one/multiple future represe...

Action Anticipation By Predicting Future Dynamic Images

Human action-anticipation methods predict what is the future action by o...

Generalizable Features From Unsupervised Learning

Humans learn a predictive model of the world and use this model to reaso...

Unsupervised Keypoint Learning for Guiding Class-Conditional Video Prediction

We propose a deep video prediction model conditioned on a single image a...

Action-conditioned Benchmarking of Robotic Video Prediction Models: a Comparative Study

A defining characteristic of intelligent systems is the ability to make ...

Learning to Anticipate Egocentric Actions by Imagination

Anticipating actions before they are executed is crucial for a wide rang...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.