A multimodal approach for multi-label movie genre classification

by   Rafael B. Mangolin, et al.

Movie genre classification is a challenging task that has increasingly attracted the attention of researchers. In this paper, we addressed the multi-label classification of the movie genres in a multimodal way. For this purpose, we created a dataset composed of trailer video clips, subtitles, synopses, and movie posters taken from 152,622 movie titles from The Movie Database. The dataset was carefully curated and organized, and it was also made available as a contribution of this work. Each movie of the dataset was labeled according to a set of eighteen genre labels. We extracted features from these data using different kinds of descriptors, namely Mel Frequency Cepstral Coefficients, Statistical Spectrum Descriptor , Local Binary Pattern with spectrograms, Long-Short Term Memory, and Convolutional Neural Networks. The descriptors were evaluated using different classifiers, such as BinaryRelevance and ML-kNN. We have also investigated the performance of the combination of different classifiers/features using a late fusion strategy, which obtained encouraging results. Based on the F-Score metric, our best result, 0.628, was obtained by the fusion of a classifier created using LSTM on the synopses, and a classifier created using CNN on movie trailer frames. When considering the AUC-PR metric, the best result, 0.673, was also achieved by combining those representations, but in addition, a classifier based on LSTM created from the subtitles was used. These results corroborate the existence of complementarity among classifiers based on different sources of information in this field of application. As far as we know, this is the most comprehensive study developed in terms of the diversity of multimedia sources of information to perform movie genre classification.


page 6

page 7

page 8

page 17


Interpretable Multi Labeled Bengali Toxic Comments Classification using Deep Learning

This paper presents a deep learning-based pipeline for categorizing Beng...

VICSOM: VIsual Clues from SOcial Media for psychological assessment

Sharing multimodal information (typically images, videos or text) in Soc...

DL based analysis of movie reviews

Undoubtedly, social media are brainstormed by a tremendous volume of sto...

Demystifying Visual Features of Movie Posters for Multi-Label Genre Identification

In the film industry, movie posters have been an essential part of adver...

Predicting Movie Genres Based on Plot Summaries

This project explores several Machine Learning methods to predict movie ...

IMDb data from two generations (1979 to 2019). Part one: Dataset

"IMDb.com" as a user-regulating and one the most-visited portal has crea...

Movie Summarization via Sparse Graph Construction

We summarize full-length movies by creating shorter videos containing th...

Please sign up or login with your details

Forgot password? Click here to reset