SDMuse: Stochastic Differential Music Editing and Generation via Hybrid Representation

11/01/2022
by   Chen Zhang, et al.
0

While deep generative models have empowered music generation, it remains a challenging and under-explored problem to edit an existing musical piece at fine granularity. In this paper, we propose SDMuse, a unified Stochastic Differential Music editing and generation framework, which can not only compose a whole musical piece from scratch, but also modify existing musical pieces in many ways, such as combination, continuation, inpainting, and style transferring. The proposed SDMuse follows a two-stage pipeline to achieve music generation and editing on top of a hybrid representation including pianoroll and MIDI-event. In particular, SDMuse first generates/edits pianoroll by iteratively denoising through a stochastic differential equation (SDE) based on a diffusion model generative prior, and then refines the generated pianoroll and predicts MIDI-event tokens auto-regressively. We evaluate the generated music of our method on ailabs1k7 pop music dataset in terms of quality and controllability on various music editing and generation tasks. Experimental results demonstrate the effectiveness of our proposed stochastic differential music editing and generation process, as well as the hybrid representations.

READ FULL TEXT

page 2

page 4

page 8

page 9

page 15

research
08/09/2023

JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models

Music generation has attracted growing interest with the advancement of ...
research
08/28/2023

InstructME: An Instruction Guided Music Edit And Remix Framework with Latent Diffusion Models

Music editing primarily entails the modification of instrument tracks or...
research
07/02/2019

Learning to Traverse Latent Spaces for Musical Score Inpainting

Music Inpainting is the task of filling in missing or lost information i...
research
12/14/2016

Imposing higher-level Structure in Polyphonic Music Generation using Convolutional Restricted Boltzmann Machines and Constraints

We introduce a method for imposing higher-level structure on generated, ...
research
10/25/2021

Actions Speak Louder than Listening: Evaluating Music Style Transfer based on Editing Experience

The subjective evaluation of music generation techniques has been mostly...
research
11/20/2018

The Effect of Explicit Structure Encoding of Deep Neural Networks for Symbolic Music Generation

With recent breakthroughs in artificial neural networks, deep generative...
research
03/20/2019

Smart Edition of MIDI Files

We address the issue of editing musical performance data, in particular ...

Please sign up or login with your details

Forgot password? Click here to reset