YouTube-8M: A Large-Scale Video Classification Benchmark

09/27/2016
by   Sami Abu-El-Haija, et al.
0

Many recent advancements in Computer Vision are attributed to large datasets. Open-source software packages for Machine Learning and inexpensive commodity hardware have reduced the barrier of entry for exploring novel approaches at scale. It is possible to train models over millions of examples within a few days. Although large-scale datasets exist for image understanding, such as ImageNet, there are no comparable size video classification datasets. In this paper, we introduce YouTube-8M, the largest multi-label video classification dataset, composed of 8 million videos (500K hours of video), annotated with a vocabulary of 4800 visual entities. To get the videos and their labels, we used a YouTube video annotation system, which labels videos with their main topics. While the labels are machine-generated, they have high-precision and are derived from a variety of human-based signals including metadata and query click signals. We filtered the video labels (Knowledge Graph entities) using both automated and manual curation strategies, including asking human raters if the labels are visually recognizable. Then, we decoded each video at one-frame-per-second, and used a Deep CNN pre-trained on ImageNet to extract the hidden representation immediately prior to the classification layer. Finally, we compressed the frame features and make both the features and video-level labels available for download. We trained various (modest) classification models on the dataset, evaluated them using popular evaluation metrics, and report them as baselines. Despite the size of the dataset, some of our models train to convergence in less than a day on a single machine using TensorFlow. We plan to release code for training a TensorFlow model and for computing metrics.

READ FULL TEXT

page 1

page 5

research
06/26/2017

An Effective Way to Improve YouTube-8M Classification Accuracy in Google Cloud Platform

Large-scale datasets have played a significant role in progress of neura...
research
06/24/2017

Encoding Video and Label Priors for Multi-label Video Classification on YouTube-8M dataset

YouTube-8M is the largest video dataset for multi-label video classifica...
research
07/13/2017

Large-scale Video Classification guided by Batch Normalized LSTM Translator

Youtube-8M dataset enhances the development of large-scale video recogni...
research
08/21/2018

Constrained-size Tensorflow Models for YouTube-8M Video Understanding Challenge

This paper presents our 10th place solution to the second YouTube-8M vid...
research
05/22/2015

Efficient Large Scale Video Classification

Video classification has advanced tremendously over the recent years. A ...
research
06/15/2017

Hierarchical Label Inference for Video Classification

Videos are a rich source of high-dimensional structured data, with a wid...
research
12/02/2019

BERT for Large-scale Video Segment Classification with Test-time Augmentation

This paper presents our approach to the third YouTube-8M video understan...

Please sign up or login with your details

Forgot password? Click here to reset