End-to-End Video Classification with Knowledge Graphs

11/06/2017
by   Fang Yuan, et al.
0

Video understanding has attracted much research attention especially since the recent availability of large-scale video benchmarks. In this paper, we address the problem of multi-label video classification. We first observe that there exists a significant knowledge gap between how machines and humans learn. That is, while current machine learning approaches including deep neural networks largely focus on the representations of the given data, humans often look beyond the data at hand and leverage external knowledge to make better decisions. Towards narrowing the gap, we propose to incorporate external knowledge graphs into video classification. In particular, we unify traditional "knowledgeless" machine learning models and knowledge graphs in a novel end-to-end framework. The framework is flexible to work with most existing video classification algorithms including state-of-the-art deep models. Finally, we conduct extensive experiments on the largest public video dataset YouTube-8M. The results are promising across the board, improving mean average precision by up to 2.9

READ FULL TEXT

page 2

page 3

research
12/14/2016

The More You Know: Using Knowledge Graphs for Image Classification

One characteristic that sets humans apart from modern learning-based com...
research
07/11/2017

Hierarchical Deep Recurrent Architecture for Video Understanding

This paper introduces the system we developed for the Youtube-8M Video U...
research
05/02/2023

Ripple Knowledge Graph Convolutional Networks For Recommendation Systems

Using knowledge graphs to assist deep learning models in making recommen...
research
07/04/2017

Aggregating Frame-level Features for Large-Scale Video Classification

This paper introduces the system we developed for the Google Cloud & You...
research
11/10/2019

A Re-evaluation of Knowledge Graph Completion Methods

Knowledge Graph Completion (KGC) aims at automatically predicting missin...
research
08/12/2017

Revisiting the Effectiveness of Off-the-shelf Temporal Modeling Approaches for Large-scale Video Classification

This paper describes our solution for the video recognition task of Acti...
research
07/18/2023

End-to-End Neural Network Training for Hyperbox-Based Classification

Hyperbox-based classification has been seen as a promising technique in ...

Please sign up or login with your details

Forgot password? Click here to reset