Predicting the Future: A Jointly Learnt Model for Action Anticipation

12/16/2019
by   Harshala Gammulle, et al.
38

Inspired by human neurological structures for action anticipation, we present an action anticipation model that enables the prediction of plausible future actions by forecasting both the visual and temporal future. In contrast to current state-of-the-art methods which first learn a model to predict future video features and then perform action anticipation using these features, the proposed framework jointly learns to perform the two tasks, future visual and temporal representation synthesis, and early action anticipation. The joint learning framework ensures that the predicted future embeddings are informative to the action anticipation task. Furthermore, through extensive experimental evaluations we demonstrate the utility of using both visual and temporal semantics of the scene, and illustrate how this representation synthesis could be achieved through a recurrent Generative Adversarial Network (GAN) framework. Our model outperforms the current state-of-the-art methods on multiple datasets: UCF101, UCF101-24, UT-Interaction and TV Human Interaction.

READ FULL TEXT

page 1

page 3

page 7

research
03/07/2020

TTPP: Temporal Transformer with Progressive Prediction for Efficient Action Anticipation

Video action anticipation aims to predict future action categories from ...
research
09/20/2019

Fine-grained Action Segmentation using the Semi-Supervised Action GAN

In this paper we address the problem of continuous fine-grained action s...
research
04/02/2022

A-ACT: Action Anticipation through Cycle Transformations

While action anticipation has garnered a lot of research interest recent...
research
11/25/2019

Forecasting Human Object Interaction: Joint Prediction of Motor Attention and Egocentric Activity

We address the challenging task of anticipating human-object interaction...
research
04/08/2019

Relational Action Forecasting

This paper focuses on multi-person action forecasting in videos. More pr...
research
11/16/2020

LAP-Net: Adaptive Features Sampling via Learning Action Progression for Online Action Detection

Online action detection is a task with the aim of identifying ongoing ac...
research
03/21/2022

Generative Adversarial Network for Future Hand Segmentation from Egocentric Video

We introduce the novel problem of anticipating a time series of future h...

Please sign up or login with your details

Forgot password? Click here to reset