On Semantic Similarity in Video Retrieval

03/18/2021
by   Michael Wray, et al.
0

Current video retrieval efforts all found their evaluation on an instance-based assumption, that only a single caption is relevant to a query video and vice versa. We demonstrate that this assumption results in performance comparisons often not indicative of models' retrieval capabilities. We propose a move to semantic similarity video retrieval, where (i) multiple videos/captions can be deemed equally relevant, and their relative ranking does not affect a method's reported performance and (ii) retrieved videos/captions are ranked by their similarity to a query. We propose several proxies to estimate semantic similarities in large-scale retrieval datasets, without additional annotations. Our analysis is performed on three commonly used video retrieval datasets (MSR-VTT, YouCook2 and EPIC-KITCHENS).

READ FULL TEXT

page 3

page 6

page 12

research
10/10/2022

Fighting FIRe with FIRE: Assessing the Validity of Text-to-Video Retrieval Benchmarks

Searching vast troves of videos with textual descriptions is a core mult...
research
04/10/2018

Imagine This! Scripts to Compositions to Videos

Imagining a scene described in natural language with realistic layout an...
research
12/27/2017

A Robust Zero-Watermark Scheme with Similarity-based Retrieval for Copyright Protection of 3D Video

The copyright protection of 3D videos has become a crucial issue. In thi...
research
01/10/2022

Multi-query Video Retrieval

Retrieving target videos based on text descriptions is a task of great p...
research
10/15/2022

Semantic Video Moments Retrieval at Scale: A New Task and a Baseline

Motivated by the increasing need of saving search effort by obtaining re...
research
03/18/2019

POI Semantic Model with a Deep Convolutional Structure

When using the electronic map, POI retrieval is the initial and importan...

Please sign up or login with your details

Forgot password? Click here to reset