DeepAI AI Chat
Log In Sign Up

Generating Music Medleys via Playing Music Puzzle Games

by   Yu-Siang Huang, et al.

Generating music medleys is about finding an optimal permutation of a given set of music clips. Toward this goal, we propose a self-supervised learning task, called the music puzzle game, to train neural network models to learn the sequential patterns in music. In essence, such a game requires machines to correctly sort a few multisecond music fragments. In the training stage, we learn the model by sampling multiple non-overlapping fragment pairs from the same songs and seeking to predict whether a given pair is consecutive and is in the correct chronological order. For testing, we design a number of puzzle games with different difficulty levels, the most difficult one being music medley, which requiring sorting fragments from different songs. On the basis of state-of-the-art Siamese convolutional network, we propose an improved architecture that learns to embed frame-level similarity scores computed from the input fragment pairs to a common space, where fragment pairs in the correct order can be more easily identified. Our result shows that the resulting model, dubbed as the similarity embedding network (SEN), performs better than competing models across different games, including music jigsaw puzzle, music sequencing, and music medley. Example results can be found at our project website,


page 4

page 7


MAP-Music2Vec: A Simple and Effective Baseline for Self-Supervised Music Audio Representation Learning

The deep learning community has witnessed an exponentially growing inter...

S3T: Self-Supervised Pre-training with Swin Transformer for Music Classification

In this paper, we propose S3T, a self-supervised pre-training method wit...

GenerationMania: Learning to Semantically Choreograph

Beatmania is a rhythm action game where players play the role of a DJ th...

Content-based Music Similarity with Triplet Networks

We explore the feasibility of using triplet neural networks to embed son...

Adaptive Music Composition for Games

The generation of music that adapts dynamically to content and actions h...

Music Classification: Beyond Supervised Learning, Towards Real-world Applications

Music classification is a music information retrieval (MIR) task to clas...

Embedding Calibration for Music Semantic Similarity using Auto-regressive Transformer

One of the advantages of using natural language processing (NLP) technol...