A Real-time Action Prediction Framework by Encoding Temporal Evolution

09/22/2017
by   Fahimeh Rezazadegan, et al.
0

Anticipating future actions is a key component of intelligence, specifically when it applies to real-time systems, such as robots or autonomous cars. While recent work has addressed prediction of raw RGB pixel values in future video frames, we focus on predicting further in future by predicting a summary of moving pixels through a sequence of frames which we call dynamic images. More precisely, given a dynamic image, we predict the motion evolution through next unseen video frames. Since this representation consists of a sequence of frames, we can go one second further into the future compared to the previous work in this field. We employed convolutional LSTMs to train our network on the dynamic images in an unsupervised learning process. Since our final goal is predicting the next action of a complex task such as an assembly task, we exploited labelled actions for the recognition process on top of predicted dynamic images. We show the effectiveness of our method on predicting the next human action in the above-mentioned task through the two-step process of predicting the next dynamic image and recognizing the action which it represents.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

page 4

page 5

page 8

03/22/2017

Predicting Deeper into the Future of Semantic Segmentation

The ability to predict and therefore to anticipate the future is an impo...
10/09/2020

Deep Sequence Learning for Video Anticipation: From Discrete and Deterministic to Continuous and Stochastic

Video anticipation is the task of predicting one/multiple future represe...
08/01/2018

Action Anticipation By Predicting Future Dynamic Images

Human action-anticipation methods predict what is the future action by o...
12/12/2016

Generalizable Features From Unsupervised Learning

Humans learn a predictive model of the world and use this model to reaso...
10/04/2019

Unsupervised Keypoint Learning for Guiding Class-Conditional Video Prediction

We propose a deep video prediction model conditioned on a single image a...
10/07/2019

Action-conditioned Benchmarking of Robotic Video Prediction Models: a Comparative Study

A defining characteristic of intelligent systems is the ability to make ...
01/13/2021

Learning to Anticipate Egocentric Actions by Imagination

Anticipating actions before they are executed is crucial for a wide rang...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.