Multitrack Music Transformer: Learning Long-Term Dependencies in Music with Diverse Instruments

07/14/2022
by   Hao-Wen Dong, et al.
0

Existing approaches for generating multitrack music with transformer models have been limited to either a small set of instruments or short music segments. This is partly due to the memory requirements of the lengthy input sequences necessitated by existing representations for multitrack music. In this work, we propose a compact representation that allows a diverse set of instruments while keeping a short sequence length. Using our proposed representation, we present the Multitrack Music Transformer (MTMT) for learning long-term dependencies in multitrack music. In a subjective listening test, our proposed model achieves competitive quality on unconditioned generation against two baseline models. We also show that our proposed model can generate samples that are twice as long as those produced by the baseline models, and, further, can do so in half the inference time. Moreover, we propose a new measure for analyzing musical self-attentions and show that the trained model learns to pay less attention to notes that form a dissonant interval with the current note, yet attending more to notes that are 4N beats away from current. Finally, our findings provide a novel foundation for future work exploring longer-form multitrack music generation and improving self-attentions for music. All source code and audio samples can be found at https://salu133445.github.io/mtmt/ .

READ FULL TEXT
research
08/18/2020

PopMAG: Pop Music Accompaniment Generation

In pop music, accompaniments are usually played by multiple instruments ...
research
07/11/2020

Transformer-XL Based Music Generation with Multiple Sequences of Time-valued Notes

Current state-of-the-art AI based classical music creation algorithms su...
research
07/13/2021

Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music

Modern keyboards allow a musician to play multiple instruments at the sa...
research
08/11/2021

Variable-Length Music Score Infilling via XLNet and Musically Specialized Positional Encoding

This paper proposes a new self-attention based model for music score inf...
research
09/04/2023

Quid Manumit – Freeing the Qubit for Art

This paper describes how to `Free the Qubit' for art, by creating standa...
research
10/06/2022

Melody Infilling with User-Provided Structural Context

This paper proposes a novel Transformer-based model for music score infi...
research
11/18/2020

Vertical-Horizontal Structured Attention for Generating Music with Chords

In this paper, we propose a lightweight music-generating model based on ...

Please sign up or login with your details

Forgot password? Click here to reset