DeepAI AI Chat
Log In Sign Up

From Benedict Cumberbatch to Sherlock Holmes: Character Identification in TV series without a Script

by   Arsha Nagrani, et al.
University of Oxford

The goal of this paper is the automatic identification of characters in TV and feature film material. In contrast to standard approaches to this task, which rely on the weak supervision afforded by transcripts and subtitles, we propose a new method requiring only a cast list. This list is used to obtain images of actors from freely available sources on the web, providing a form of partial supervision for this task. In using images of actors to recognize characters, we make the following three contributions: (i) We demonstrate that an automated semi-supervised learning approach is able to adapt from the actor's face to the character's face, including the face context of the hair; (ii) By building voice models for every character, we provide a bridge between frontal faces (for which there is plenty of actor-level supervision) and profile (for which there is very little or none); and (iii) by combining face context and speaker identification, we are able to identify characters with partially occluded faces and extreme facial poses. Results are presented on the TV series 'Sherlock' and the feature film 'Casablanca'. We achieve the state-of-the-art on the Casablanca benchmark, surpassing previous methods that have used the stronger supervision available from transcripts.


page 2

page 3

page 6

page 9


Who's that Actor? Automatic Labelling of Actors in TV series starting from IMDB Images

In this work, we aim at automatically labeling actors in a TV series. Ra...

TVShowGuess: Character Comprehension in Stories as Speaker Guessing

We propose a new task for assessing machines' skills of understanding fi...

Audio visual character profiles for detecting background characters in entertainment media

An essential goal of computational media intelligence is to support unde...

Deep Multimodal Speaker Naming

Automatic speaker naming is the problem of localizing as well as identif...

Character-focused Video Thumbnail Retrieval

We explore retrieving character-focused video frames as candidates for b...

Remembering winter was coming

Today's popular TV series tend to develop continuous, complex plots span...

Video Face Clustering with Unknown Number of Clusters

Understanding videos such as TV series and movies requires analyzing who...