VRFP: On-the-fly Video Retrieval using Web Images and Fast Fisher Vector Products

by   Xintong Han, et al.

VRFP is a real-time video retrieval framework based on short text input queries, which obtains weakly labeled training images from the web after the query is known. The retrieved web images representing the query and each database video are treated as unordered collections of images, and each collection is represented using a single Fisher Vector built on CNN features. Our experiments show that a Fisher Vector is robust to noise present in web images and compares favorably in terms of accuracy to other standard representations. While a Fisher Vector can be constructed efficiently for a new query, matching against the test set is slow due to its high dimensionality. To perform matching in real-time, we present a lossless algorithm that accelerates the inner product computation between high dimensional Fisher Vectors. We prove that the expected number of multiplications required decreases quadratically with the sparsity of Fisher Vectors. We are not only able to construct and apply query models in real-time, but with the help of a simple re-ranking scheme, we also outperform state-of-the-art automatic retrieval methods by a significant margin on TRECVID MED13 (3.5 (5.2 different paradigms for automatic video retrieval - zero-shot learning and on-the-fly retrieval.



There are no comments yet.


page 11

page 13


GPU-FV: Realtime Fisher Vector and Its Applications in Video Monitoring

Fisher vector has been widely used in many multimedia retrieval and visu...

Efficient On-the-fly Category Retrieval using ConvNets and GPUs

We investigate the gains in precision and speed, that can be obtained by...

Deep Tensor Encoding

Learning an encoding of feature vectors in terms of an over-complete dic...

Siamese Network of Deep Fisher-Vector Descriptors for Image Retrieval

This paper addresses the problem of large scale image retrieval, with th...

Multi-query Video Retrieval

Retrieving target videos based on text descriptions is a task of great p...

Cross-Media Similarity Evaluation for Web Image Retrieval in the Wild

In order to retrieve unlabeled images by textual queries, cross-media si...

Towards Accurate Camera Geopositioning by Image Matching

In this work, we present a camera geopositioning system based on matchin...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.