MusicTM-Dataset for Joint Representation Learning among Sheet Music, Lyrics, and Musical Audio

12/01/2020
by   Donghuo Zeng, et al.
0

This work present a music dataset named MusicTM-Dataset, which is utilized in improving the representation learning ability of different types of cross-modal retrieval (CMR). Little large music dataset including three modalities is available for learning representations for CMR. To collect a music dataset, we expand the original musical notation to synthesize audio and generated sheet-music image, and build musical notation based sheet-music image, audio clip and syllable-denotation text as fine-grained alignment, such that the MusicTM-Dataset can be exploited to receive shared representation for multimodal data points. The MusicTM-Dataset presents 3 kinds of modalities, which consists of the image of sheet-music, the text of lyrics and synthesized audio, their representations are extracted by some advanced models. In this paper, we introduce the background of music dataset and express the process of our data collection. Based on our dataset, we achieve some basic methods for CMR tasks. The MusicTM-Dataset are accessible in https: //github.com/dddzeng/MusicTM-Dataset.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

08/10/2019

Audio-Visual Embedding for Cross-Modal MusicVideo Retrieval through Supervised Deep CCA

Deep learning has successfully shown excellent performance in learning j...
07/29/2020

Unsupervised Generative Adversarial Alignment Representation for Sheet music, Audio and Lyrics

Sheet music, audio, and lyrics are three main modalities during writing ...
01/11/2022

Music2Video: Automatic Generation of Music Video with fusion of audio and text

Creation of images using generative adversarial networks has been widely...
07/29/2020

dMelodies: A Music Dataset for Disentanglement Learning

Representation learning focused on disentangling the underlying factors ...
11/11/2018

PerformanceNet: Score-to-Audio Music Generation with Multi-Band Convolutional Residual Network

Music creation is typically composed of two parts: composing the musical...
11/08/2019

Automatic Identification of Traditional Colombian Music Genres based on Audio Content Analysis and Machine Learning Technique

Colombia has a diversity of genres in traditional music, which allows to...
05/21/2021

LoopNet: Musical Loop Synthesis Conditioned On Intuitive Musical Parameters

Loops, seamlessly repeatable musical segments, are a cornerstone of mode...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.