Self-Regulated Learning for Egocentric Video Activity Anticipation

11/23/2021
by   Zhaobo Qi, et al.
0

Future activity anticipation is a challenging problem in egocentric vision. As a standard future activity anticipation paradigm, recursive sequence prediction suffers from the accumulation of errors. To address this problem, we propose a simple and effective Self-Regulated Learning framework, which aims to regulate the intermediate representation consecutively to produce representation that (a) emphasizes the novel information in the frame of the current time-stamp in contrast to previously observed content, and (b) reflects its correlation with previously observed frames. The former is achieved by minimizing a contrastive loss, and the latter can be achieved by a dynamic reweighing mechanism to attend to informative frames in the observed content with a similarity comparison between feature of the current frame and observed frames. The learned final video representation can be further enhanced by multi-task learning which performs joint feature learning on the target activity labels and the automatically detected action and object class tokens. SRL sharply outperforms existing state-of-the-art in most cases on two egocentric video datasets and two third-person video datasets. Its effectiveness is also verified by the experimental fact that the action and object concepts that support the activity semantics can be accurately identified.

READ FULL TEXT

page 12

page 16

research
11/11/2020

Unsupervised Video Representation Learning by Bidirectional Feature Prediction

This paper introduces a novel method for self-supervised video represent...
research
10/28/2020

Cycle-Contrast for Self-Supervised Video Representation Learning

We present Cycle-Contrastive Learning (CCL), a novel self-supervised met...
research
08/10/2023

Is there progress in activity progress prediction?

Activity progress prediction aims to estimate what percentage of an acti...
research
04/20/2023

Search-Map-Search: A Frame Selection Paradigm for Action Recognition

Despite the success of deep learning in video understanding tasks, proce...
research
08/02/2019

Captioning Near-Future Activity Sequences

Most of the existing works on human activity analysis focus on recogniti...
research
08/13/2018

Time Perception Machine: Temporal PointProcesses for the When, Where and What ofActivity Prediction

Numerous powerful point process models have been developed to understand...
research
11/24/2018

Discriminative Feature Learning for Unsupervised Video Summarization

In this paper, we address the problem of unsupervised video summarizatio...

Please sign up or login with your details

Forgot password? Click here to reset