Learning the Predictability of the Future

01/01/2021
by   Dídac Surís, et al.
10

We introduce a framework for learning from unlabeled video what is predictable in the future. Instead of committing up front to features to predict, our approach learns from data which features are predictable. Based on the observation that hyperbolic geometry naturally and compactly encodes hierarchical structure, we propose a predictive model in hyperbolic space. When the model is most confident, it will predict at a concrete level of the hierarchy, but when the model is not confident, it learns to automatically select a higher level of abstraction. Experiments on two established datasets show the key role of hierarchical representations for action prediction. Although our representation is trained with unlabeled video, visualizations show that action hierarchies emerge in the representation.

READ FULL TEXT

Authors

page 1

page 5

page 7

page 8

08/20/2020

Learning to Abstract and Predict Human Actions

Human activities are naturally structured as hierarchies unrolled over t...
03/12/2019

Unsupervised Discovery of Parts, Structure, and Dynamics

Humans easily recognize object parts and their hierarchical structure by...
04/29/2015

Anticipating Visual Representations from Unlabeled Video

Anticipating actions and objects before they start or appear is a diffic...
06/28/2020

Video Representations of Goals Emerge from Watching Failure

We introduce a video representation learning framework that models the l...
01/25/2019

A Neurally-Inspired Hierarchical Prediction Network for Spatiotemporal Sequence Learning and Prediction

In this paper we developed a hierarchical network model, called Hierarch...
10/21/2021

Variational Predictive Routing with Nested Subjective Timescales

Discovery and learning of an underlying spatiotemporal hierarchy in sequ...
10/09/2020

Deep Sequence Learning for Video Anticipation: From Discrete and Deterministic to Continuous and Stochastic

Video anticipation is the task of predicting one/multiple future represe...

Code Repositories

hyperfuture

Code for the paper Learning the Predictability of the Future


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.