Creating A Multi-track Classical Musical Performance Dataset for Multimodal Music Analysis: Challenges, Insights, and Applications

12/27/2016
by   Bochen Li, et al.
0

We introduce a dataset for facilitating audio-visual analysis of musical performances. The dataset comprises a number of simple multi-instrument classical music pieces assembled from coordinated but separately recorded performances of individual tracks. For each piece, we provide the musical score in MIDI format, the audio recordings of the individual tracks, the audio and video recording of the assembled mixture, and ground-truth annotation files including frame-level and note-level transcriptions. We describe our methodology for the creation of this dataset, particularly highlighting our approaches for addressing the challenges involved in maintaining synchronization and naturalness. We compare this dataset with existing widely used music audio datasets on the synchronization quality and show its high quality. We anticipate that the dataset will be useful for the development and evaluation of many existing music information retrieval (MIR) tasks, as well as many novel multi-modal tasks. On this end, we benchmark this dataset with existing music audio datasets using two existing MIR tasks (multi-pitch analysis and score-informed source separation). We also define two novel multi-modal MIR tasks (visually informed multi-pitch analysis and polyphonic vibrato analysis), provide evaluation measures and baseline systems for future comparisons. Finally, we propose several emerging research directions that can be supported by this dataset.

READ FULL TEXT

page 6

page 9

research
06/14/2020

Solos: A Dataset for Audio-Visual Music Analysis

In this paper, we present a new dataset of music performance videos whic...
research
09/18/2023

HumTrans: A Novel Open-Source Dataset for Humming Melody Transcription and Beyond

This paper introduces the HumTrans dataset, which is publicly available ...
research
10/06/2022

AnimeTAB: A new guitar tablature dataset of anime and game music

While guitar tablature has become a popular topic in MIR research, there...
research
09/14/2022

CCOM-HuQin: an Annotated Multimodal Chinese Fiddle Performance Dataset

HuQin is a family of traditional Chinese bowed string instruments. Playi...
research
11/21/2022

A Dataset for Greek Traditional and Folk Music: Lyra

Studying under-represented music traditions under the MIR scope is cruci...
research
08/14/2017

Creating an A Cappella Singing Audio Dataset for Automatic Jingju Singing Evaluation Research

The data-driven computational research on automatic jingju (also known a...
research
06/12/2018

The NES Music Database: A multi-instrumental dataset with expressive performance attributes

Existing research on music generation focuses on composition, but often ...

Please sign up or login with your details

Forgot password? Click here to reset