Multi-attention Networks for Temporal Localization of Video-level Labels

11/15/2019
by   Lijun Zhang, et al.
1

Temporal localization remains an important challenge in video understanding. In this work, we present our solution to the 3rd YouTube-8M Video Understanding Challenge organized by Google Research. Participants were required to build a segment-level classifier using a large-scale training data set with noisy video-level labels and a relatively small-scale validation data set with accurate segment-level labels. We formulated the problem as a multiple instance multi-label learning and developed an attention-based mechanism to selectively emphasize the important frames by attention weights. The model performance is further improved by constructing multiple sets of attention networks. We further fine-tuned the model using the segment-level data set. Our final model consists of an ensemble of attention/multi-attention networks, deep bag of frames models, recurrent neural networks and convolutional neural networks. It ranked 13th on the private leader board and stands out for its efficient usage of resources.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

06/16/2017

The Monkeytyping Solution to the YouTube-8M Video Understanding Challenge

This article describes the final solution of team monkeytyping, who fini...
12/02/2019

BERT for Large-scale Video Segment Classification with Test-time Augmentation

This paper presents our approach to the third YouTube-8M video understan...
09/12/2018

Label Denoising with Large Ensembles of Heterogeneous Neural Networks

Despite recent advances in computer vision based on various convolutiona...
07/13/2017

UTS submission to Google YouTube-8M Challenge 2017

In this paper, we present our solution to Google YouTube-8M Video Classi...
07/11/2017

Hierarchical Deep Recurrent Architecture for Video Understanding

This paper introduces the system we developed for the Youtube-8M Video U...
03/18/2021

Enhancing Transformer for Video Understanding Using Gated Multi-Level Attention and Temporal Adversarial Training

The introduction of Transformer model has led to tremendous advancements...
01/18/2019

Adapting Convolutional Neural Networks for Geographical Domain Shift

We present the winning solution for the Inclusive Images Competition org...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.