Temporal-Needle: A view and appearance invariant video descriptor

12/14/2016
by   Michal Yarom, et al.
0

The ability to detect similar actions across videos can be very useful for real-world applications in many fields. However, this task is still challenging for existing systems, since videos that present the same action, can be taken from significantly different viewing directions, performed by different actors and backgrounds and under various video qualities. Video descriptors play a significant role in these systems. In this work we propose the "temporal-needle" descriptor which captures the dynamic behavior, while being invariant to viewpoint and appearance. The descriptor is computed using multi temporal scales of the video and by computing self-similarity for every patch through time in every temporal scale. The descriptor is computed for every pixel in the video. However, to find similar actions across videos, we consider only a small subset of the descriptors - the statistical significant descriptors. This allow us to find good correspondences across videos more efficiently. Using the descriptor, we were able to detect the same behavior across videos in a variety of scenarios. We demonstrate the use of the descriptor in tasks such as temporal and spatial alignment, action detection and even show its potential in unsupervised video clustering into categories. In this work we handled only videos taken with stationary cameras, but the descriptor can be extended to handle moving camera as well.

READ FULL TEXT

page 8

page 11

page 12

page 13

page 14

page 15

page 16

page 18

research
06/08/2015

Circulant temporal encoding for video retrieval and temporal alignment

We address the problem of specific video event retrieval. Given a query ...
research
12/03/2014

Event Retrieval Using Motion Barcodes

We introduce a simple and effective method for retrieval of videos showi...
research
09/30/2021

Deep Learning-based Action Detection in Untrimmed Videos: A Survey

Understanding human behavior and activity facilitates advancement of num...
research
11/30/2015

Behavior Discovery and Alignment of Articulated Object Classes from Unstructured Video

We propose an automatic system for organizing the content of a collectio...
research
08/07/2023

Spatialyze: A Geospatial Video Analytics System with Spatial-Aware Optimizations

Videos that are shot using commodity hardware such as phones and surveil...
research
02/13/2015

Long-short Term Motion Feature for Action Classification and Retrieval

We propose a method for representing motion information for video classi...
research
11/20/2019

A Human Action Descriptor Based on Motion Coordination

In this paper, we present a descriptor for human whole-body actions base...

Please sign up or login with your details

Forgot password? Click here to reset