Rethinking movie genre classification with fine-grained semantic clustering

by   Edward Fish, et al.
University of Surrey

Movie genre classification is an active research area in machine learning. However, due to the limited labels available, there can be large semantic variations between movies within a single genre definition. We expand these 'coarse' genre labels by identifying 'fine-grained' semantic information within the multi-modal content of movies. By leveraging pre-trained 'expert' networks, we learn the influence of different combinations of modes for multi-label genre classification. Using a contrastive loss, we continue to fine-tune this 'coarse' genre classification network to identify high-level intertextual similarities between the movies across all genre labels. This leads to a more 'fine-grained' and detailed clustering, based on semantic similarities while still retaining some genre information. Our approach is demonstrated on a newly introduced multi-modal 37,866,450 frame, 8,800 movie trailer dataset, MMX-Trailer-20, which includes pre-computed audio, location, motion, and image embeddings.


page 2

page 3

page 5

page 8

page 10

page 12


Semantic Bilinear Pooling for Fine-Grained Recognition

Fine-grained recognition, e.g., vehicle identification or bird classific...

Contrastive Multi-Modal Clustering

Multi-modal clustering, which explores complementary information from mu...

Contrastive Bootstrapping for Label Refinement

Traditional text classification typically categorizes texts into pre-def...

Evaluating Word Embeddings in Multi-label Classification Using Fine-grained Name Typing

Embedding models typically associate each word with a single real-valued...

An Efficient COarse-to-fiNE Alignment Framework @ Ego4D Natural Language Queries Challenge 2022

This technical report describes the CONE approach for Ego4D Natural Lang...

Efficient Algorithms for Learning from Coarse Labels

For many learning problems one may not have access to fine grained label...

Please sign up or login with your details

Forgot password? Click here to reset