Musical Speech: A Transformer-based Composition Tool

08/02/2021
by   Jason d'Eon, et al.
0

In this paper, we propose a new compositional tool that will generate a musical outline of speech recorded/provided by the user for use as a musical building block in their compositions. The tool allows any user to use their own speech to generate musical material, while still being able to hear the direct connection between their recorded speech and the resulting music. The tool is built on our proposed pipeline. This pipeline begins with speech-based signal processing, after which some simple musical heuristics are applied, and finally these pre-processed signals are passed through Transformer models trained on new musical tasks. We illustrate the effectiveness of our pipeline – which does not require a paired dataset for training – through examples of music created by musicians making use of our tool.

READ FULL TEXT

page 8

page 11

page 21

research
01/06/2023

Multi-Genre Music Transformer – Composing Full Length Musical Piece

In the task of generating music, the art factor plays a big role and is ...
research
10/19/2022

Modeling Animal Vocalizations through Synthesizers

Modeling real-world sound is a fundamental problem in the creative use o...
research
06/18/2021

Synchronising speech segments with musical beats in Mandarin and English singing

Generating synthesised singing voice with models trained on speech data ...
research
07/04/2022

An adaptive music generation architecture for games based on the deep learning Transformer mode

This paper presents an architecture for generating music for video games...
research
08/26/2022

Mel Spectrogram Inversion with Stable Pitch

Vocoders are models capable of transforming a low-dimensional spectral r...
research
10/06/2022

Melody Infilling with User-Provided Structural Context

This paper proposes a novel Transformer-based model for music score infi...
research
05/06/2019

Topology of Networks in Generalized Musical Spaces

The abstraction of musical structures (notes, melodies, chords, harmonic...

Please sign up or login with your details

Forgot password? Click here to reset