HLVU : A New Challenge to Test Deep Understanding of Movies the Way Humans do

05/01/2020
by   Keith Curtis, et al.
0

In this paper we propose a new evaluation challenge and direction in the area of High-level Video Understanding. The challenge we are proposing is designed to test automatic video analysis and understanding, and how accurately systems can comprehend a movie in terms of actors, entities, events and their relationship to each other. A pilot High-Level Video Understanding (HLVU) dataset of open source movies were collected for human assessors to build a knowledge graph representing each of them. A set of queries will be derived from the knowledge graph to test systems on retrieving relationships among actors, as well as reasoning and retrieving non-visual concepts. The objective is to benchmark if a computer system can "understand" non-explicit but obvious relationships the same way humans do when they watch the same movies. This is long-standing problem that is being addressed in the text domain and this project moves similar research to the video domain. Work of this nature is foundational to future video analytics and video understanding technologies. This work can be of interest to streaming services and broadcasters hoping to provide more intuitive ways for their customers to interact with and consume video content.

READ FULL TEXT
research
11/19/2022

A Unified Model for Video Understanding and Knowledge Embedding with Heterogeneous Knowledge Graph Dataset

Video understanding is an important task in short video business platfor...
research
04/02/2021

Visual Semantic Role Labeling for Video Understanding

We propose a new framework for understanding and representing related sa...
research
02/27/2022

Concept Graph Neural Networks for Surgical Video Understanding

We constantly integrate our knowledge and understanding of the world to ...
research
06/23/2016

VideoMCC: a New Benchmark for Video Comprehension

While there is overall agreement that future technology for organizing, ...
research
03/25/2019

Video Relationship Reasoning using Gated Spatio-Temporal Energy Graph

Visual relationship reasoning is a crucial yet challenging task for unde...
research
07/02/2018

Deep Reasoning with Knowledge Graph for Social Relationship Understanding

Social relationships (e.g., friends, couple etc.) form the basis of the ...
research
04/01/2022

From Data to Knowledge Graphs: A Multi-Layered Method to Model User's Visual Analytics Workflow for Analytical Purposes

The importance of knowledge generation drives much of Visual Analytics (...

Please sign up or login with your details

Forgot password? Click here to reset