Online Multi-modal Person Search in Videos

08/08/2020
by   Jiangyue Xia, et al.
3

The task of searching certain people in videos has seen increasing potential in real-world applications, such as video organization and editing. Most existing approaches are devised to work in an offline manner, where identities can only be inferred after an entire video is examined. This working manner precludes such methods from being applied to online services or those applications that require real-time responses. In this paper, we propose an online person search framework, which can recognize people in a video on the fly. This framework maintains a multimodal memory bank at its heart as the basis for person recognition, and updates it dynamically with a policy obtained by reinforcement learning. Our experiments on a large movie dataset show that the proposed method is effective, not only achieving remarkable improvements over online schemes but also outperforming offline methods.

READ FULL TEXT

page 13

page 14

06/03/2021

APES: Audiovisual Person Search in Untrimmed Video

Humans are arguably one of the most important subjects in video streams,...
07/27/2018

Person Search in Videos with One Portrait Through Visual and Temporal Links

In real-world applications, e.g. law enforcement and video retrieval, on...
09/17/2020

Multi-modal Summarization for Video-containing Documents

Summarization of multimedia data becomes increasingly significant as it ...
07/02/2019

Dynamic Face Video Segmentation via Reinforcement Learning

For real-time semantic video segmentation, most recent works utilise a d...
12/11/2020

A Multi-task Joint Framework for Real-time Person Search

Person search generally involves three important parts: person detection...
05/08/2018

FFNet: Video Fast-Forwarding via Reinforcement Learning

For many applications with limited computation, communication, storage a...
08/08/2020

A Unified Framework for Shot Type Classification Based on Subject Centric Lens

Shots are key narrative elements of various videos, e.g. movies, TV seri...