Evaluating Temporal Queries Over Video Feeds

03/02/2020
by   Yueting Chen, et al.
0

Recent advances in Computer Vision and Deep Learning made possible the efficient extraction of a schema from frames of streaming video. As such, a stream of objects and their associated classes along with unique object identifiers derived via object tracking can be generated, providing unique objects as they are captured across frames. In this paper we initiate a study of temporal queries involving objects and their co-occurrences in video feeds. For example, queries that identify video segments during which the same two red cars and the same two humans appear jointly for five minutes are of interest to many applications ranging from law enforcement to security and safety. We take the first step and define such queries in a way that they incorporate certain physical aspects of video capture such as object occlusion. We present an architecture consisting of three layers, namely object detection/tracking, intermediate data generation and query evaluation. We propose two techniques,MFS and SSG, to organize all detected objects in the intermediate data generation layer, which effectively, given the queries, minimizes the number of objects and frames that have to be considered during query evaluation. We also introduce an algorithm called State Traversal (ST) that processes incoming frames against the SSG and efficiently prunes objects and frames unrelated to query evaluation, while maintaining all states required for succinct query evaluation. We present the results of a thorough experimental evaluation utilizing both real and synthetic data establishing the trade-offs between MFS and SSG. We stress various parameters of interest in our evaluation and demonstrate that the proposed query evaluation methodology coupled with the proposed algorithms is capable to evaluate temporal queries over video feeds efficiently, achieving orders of magnitude performance benefits.

READ FULL TEXT
research
02/24/2020

Video Monitoring Queries

Recent advances in video processing utilizing deep learning primitives a...
research
10/28/2018

Deep Affinity Network for Multiple Object Tracking

Multiple Object Tracking (MOT) plays an important role in solving many f...
research
06/14/2018

Semantic Image Retrieval by Uniting Deep Neural Networks and Cognitive Architectures

Image and video retrieval by their semantic content has been an importan...
research
05/19/2020

ExSample: Efficient Searches on Video Repositories through Adaptive Sampling

Capturing and processing video is increasingly common as cameras become ...
research
03/07/2017

NoScope: Optimizing Neural Network Queries over Video at Scale

Recent advances in computer vision-in the form of deep neural networks-h...
research
05/18/2021

Reachability and Top-k Reachability Queries with Transfer Decay

The prevalence of location tracking systems has resulted in large volume...
research
01/19/2022

Tracking Where Events Take Place: Reverse Spatial Term Queries on Streaming Data

A large volume of content generated by online users is geo-tagged and th...

Please sign up or login with your details

Forgot password? Click here to reset