DeepAI AI Chat
Log In Sign Up

Pop Music Highlighter: Marking the Emotion Keypoints

by   Yu-Siang Huang, et al.

The goal of music highlight extraction is to get a short consecutive segment of a piece of music that provides an effective representation of the whole piece. In a previous work, we introduced an attention-based convolutional recurrent neural network that uses music emotion classification as a surrogate task for music highlight extraction, for Pop songs. The rationale behind that approach is that the highlight of a song is usually the most emotional part. This paper extends our previous work in the following two aspects. First, methodology-wise we experiment with a new architecture that does not need any recurrent layers, making the training process faster. Moreover, we compare a late-fusion variant and an early-fusion variant to study which one better exploits the attention mechanism. Second, we conduct and report an extensive set of experiments comparing the proposed attention-based methods against a heuristic energy-based method, a structural repetition-based method, and a few other simple feature-based methods for this task. Due to the lack of public-domain labeled data for highlight extraction, following our previous work we use the RWC POP 100-song data set to evaluate how the detected highlights overlap with any chorus sections of the songs. The experiments demonstrate the effectiveness of our methods over competing methods. For reproducibility, we open source the code and pre-trained model at


Automatic Music Highlight Extraction using Convolutional Recurrent Attention Networks

Music highlights are valuable contents for music services. Most methods ...

musicnn: Pre-trained convolutional neural networks for music audio tagging

Pronounced as "musician", the musicnn library contains a set of pre-trai...

AttendAffectNet: Self-Attention based Networks for Predicting Affective Responses from Movies

In this work, we propose different variants of the self-attention based ...

ADFF: Attention Based Deep Feature Fusion Approach for Music Emotion Recognition

Music emotion recognition (MER), a sub-task of music information retriev...

Singer Identification Using Deep Timbre Feature Learning with KNN-Net

In this paper, we study the issue of automatic singer identification (SI...

Constructing a Highlight Classifier with an Attention Based LSTM Neural Network

Data is being produced in larger quantities than ever before in human hi...

Code Repositories


code for music highlight extraction

view repo