Temporal Context Aggregation for Video Retrieval with Contrastive Learning

08/04/2020
by   Jie Shao, et al.
11

The current research focus on Content-Based Video Retrieval requires higher-level video representation describing the long-range semantic dependencies of relevant incidents, events, etc. However, existing methods commonly process the frames of a video as individual images or short clips, making the modeling of long-range semantic dependencies difficult. In this paper, we propose TCA (Temporal Context Aggregation for Video Retrieval), a video representation learning framework that incorporates long-range temporal information between frame-level features using the self-attention mechanism. To train it on video retrieval datasets, we propose a supervised contrastive learning method that performs automatic hard negative mining and utilizes the memory bank mechanism to increase the capacity of negative samples. Extensive experiments are conducted on multiple video retrieval tasks, such as CC_WEB_VIDEO, FIVR-200K, and EVVE. The proposed method shows a significant performance advantage ( 17 with video-level features, and deliver competitive results with 22x faster inference time comparing with frame-level features.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/17/2021

Temporal Memory Attention for Video Semantic Segmentation

Video semantic segmentation requires to utilize the complex temporal rel...
research
05/18/2022

VRAG: Region Attention Graphs for Content-Based Video Retrieval

Content-based Video Retrieval (CBVR) is used on media-sharing platforms ...
research
02/19/2020

Unsupervised Temporal Feature Aggregation for Event Detection in Unstructured Sports Videos

Image-based sports analytics enable automatic retrieval of key events in...
research
04/16/2021

Self-supervised Video Retrieval Transformer Network

Content-based video retrieval aims to find videos from a large video dat...
research
05/20/2023

Joint Generative-Contrastive Representation Learning for Anomalous Sound Detection

In this paper, we propose a joint generative and contrastive representat...
research
04/06/2022

ECLIPSE: Efficient Long-range Video Retrieval using Sight and Sound

We introduce an audiovisual method for long-range text-to-video retrieva...
research
09/04/2020

Video Moment Retrieval via Natural Language Queries

In this paper, we propose a novel method for video moment retrieval (VMR...

Please sign up or login with your details

Forgot password? Click here to reset