A Convolutional-Attentional Neural Framework for Structure-Aware Performance-Score Synchronization

by   Ruchit Agrawal, et al.

Performance-score synchronization is an integral task in signal processing, which entails generating an accurate mapping between an audio recording of a performance and the corresponding musical score. Traditional synchronization methods compute alignment using knowledge-driven and stochastic approaches, and are typically unable to generalize well to different domains and modalities. We present a novel data-driven method for structure-aware performance-score synchronization. We propose a convolutional-attentional architecture trained with a custom loss based on time-series divergence. We conduct experiments for the audio-to-MIDI and audio-to-image alignment tasks pertained to different score modalities. We validate the effectiveness of our method via ablation studies and comparisons with state-of-the-art alignment approaches. We demonstrate that our approach outperforms previous synchronization methods for a variety of test settings across score modalities and acoustic conditions. Our method is also robust to structural differences between the performance and score sequences, which is a common limitation of standard alignment approaches.



page 1


Towards Context-Aware Neural Performance-Score Synchronisation

Music can be represented in multiple forms, such as in the audio form as...

Learning Frame Similarity using Siamese networks for Audio-to-Score Alignment

Audio-to-score alignment aims at generating an accurate mapping between ...

Structure-Aware Audio-to-Score Alignment using Progressively Dilated Convolutional Neural Networks

The identification of structural differences between a music performance...

Rethinking Evaluation Methodology for Audio-to-Score Alignment

This paper offers a precise, formal definition of an audio-to-score alig...

Multi-Frequency Phase Synchronization

We propose a novel formulation for phase synchronization -- the statisti...

Are you doing what I say? On modalities alignment in ALFRED

ALFRED is a recently proposed benchmark that requires a model to complet...

A Synchronization Algorithm Based on Moving Average for Robust Audio Watermarking Scheme

A synchronization code scheme based on moving average is proposed for ro...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.