ECO: Efficient Convolutional Network for Online Video Understanding

04/24/2018
by   Mohammadreza Zolfaghari, et al.
0

The state of the art in video understanding suffers from two problems: (1) The major part of reasoning is performed locally in the video, therefore, it misses important relationships within actions that span several seconds. (2) While there are local methods with fast per-frame processing, the processing of the whole video is not efficient and hampers fast video retrieval or online classification of long-term activities. In this paper, we introduce a network architecture that takes long-term content into account and enables fast per-video processing at the same time. The architecture is based on merging long-term content already in the network rather than in a post-hoc fusion. Together with a sampling strategy, which exploits that neighboring frames are largely redundant, this yields high-quality action classification and video captioning at up to 230 videos per second, where each video can consist of a few hundred frames. The approach achieves competitive performance across all datasets while being 10x to 80x faster than state-of-the-art methods.

READ FULL TEXT

Authors

page 12

page 14

12/12/2018

Long-Term Feature Banks for Detailed Video Understanding

To understand the world, we humans constantly need to relate the present...
04/03/2018

When will you do what? - Anticipating Temporal Occurrences of Activities

Analyzing human actions in videos has gained increased attention recentl...
01/20/2022

MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition

While today's video recognition systems parse snapshots or short clips a...
04/26/2019

ARCHANGEL: Tamper-proofing Video Archives using Temporal Content Hashes on the Blockchain

We present ARCHANGEL; a novel distributed ledger based system for assuri...
10/15/2019

Tiny Video Networks

Video understanding is a challenging problem with great impact on the ab...
05/08/2018

FFNet: Video Fast-Forwarding via Reinforcement Learning

For many applications with limited computation, communication, storage a...
10/13/2020

Video Action Understanding: A Tutorial

Many believe that the successes of deep learning on image understanding ...

Code Repositories

ECO-efficient-video-understanding

Code and models of paper " ECO: Efficient Convolutional Network for Online Video Understanding"


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.