DeepAI AI Chat
Log In Sign Up

Finding Islands of Predictability in Action Forecasting

by   Daniel Scarafoni, et al.

We address dense action forecasting: the problem of predicting future action sequence over long durations based on partial observation. Our key insight is that future action sequences are more accurately modeled with variable, rather than one, levels of abstraction, and that the optimal level of abstraction can be dynamically selected during the prediction process. Our experiments show that most parts of future action sequences can be predicted confidently in fine detail only in small segments of future frames, which are effectively “islands” of high model prediction confidence in a “sea” of uncertainty. We propose a combination Bayesian neural network and hierarchical convolutional segmentation model to both accurately predict future actions and optimally select abstraction levels. We evaluate this approach on standard datasets against existing state-of-the-art systems and demonstrate that our “islands of predictability” approach maintains fine-grained action predictions while also making accurate abstract predictions where systems were previously unable to do so, and thus results in substantial, monotonic increases in accuracy.


Learning to Abstract and Predict Human Actions

Human activities are naturally structured as hierarchies unrolled over t...

Forecasting Future Sequence of Actions to Complete an Activity

Future human action forecasting from partial observations of activities ...

Learning the Predictability of the Future

We introduce a framework for learning from unlabeled video what is predi...

Deep Sequence Learning for Video Anticipation: From Discrete and Deterministic to Continuous and Stochastic

Video anticipation is the task of predicting one/multiple future represe...

PLAN-B: Predicting Likely Alternative Next Best Sequences for Action Prediction

Action prediction focuses on anticipating actions before they happen. Re...

Forecasting Future Action Sequences with Neural Memory Networks

We propose a novel neural memory network based framework for future acti...

Fine-Grained Address Segmentation for Attention-Based Variable-Degree Prefetching

Machine learning algorithms have shown potential to improve prefetching ...