Audio-Conditioned U-Net for Position Estimation in Full Sheet Images

10/16/2019
by   Florian Henkel, et al.
0

The goal of score following is to track a musical performance, usually in the form of audio, in a corresponding score representation. Established methods mainly rely on computer-readable scores in the form of MIDI or MusicXML and achieve robust and reliable tracking results. Recently, multimodal deep learning methods have been used to follow along musical performances in raw sheet images. Among the current limits of these systems is that they require a non trivial amount of preprocessing steps that unravel the raw sheet image into a single long system of staves. The current work is an attempt at removing this particular limitation. We propose an architecture capable of estimating matching score positions directly within entire unprocessed sheet images. We argue that this is a necessary first step towards a fully integrated score following system that does not rely on any preprocessing steps such as optical music recognition.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/17/2018

Learning to Listen, Read, and Follow: Score Following as a Reinforcement Learning Game

Score following is the process of tracking a musical performance (audio)...
research
07/21/2020

Learning to Read and Follow Music in Complete Score Sheet Images

This paper addresses the task of score following in sheet music given as...
research
05/06/2022

Musical Score Following and Audio Alignment

Real-time tracking of the position of a musical performance on a musical...
research
08/07/2017

Aktuelle Entwicklungen in der Automatischen Musikverfolgung

In this paper we present current trends in real-time music tracking (a.k...
research
05/10/2021

Multi-modal Conditional Bounding Box Regression for Music Score Following

This paper addresses the problem of sheet-image-based on-line audio-to-s...
research
03/20/2017

Dance Dance Convolution

Dance Dance Revolution (DDR) is a popular rhythm-based video game. Playe...
research
07/21/2017

HMM-based Writer Identification in Music Score Documents without Staff-Line Removal

Writer identification from musical score documents is a challenging task...

Please sign up or login with your details

Forgot password? Click here to reset