DeepAI AI Chat
Log In Sign Up

Differentiable Digital Signal Processing Mixture Model for Synthesis Parameter Extraction from Mixture of Harmonic Sounds

by   Masaya Kawamura, et al.

A differentiable digital signal processing (DDSP) autoencoder is a musical sound synthesizer that combines a deep neural network (DNN) and spectral modeling synthesis. It allows us to flexibly edit sounds by changing the fundamental frequency, timbre feature, and loudness (synthesis parameters) extracted from an input sound. However, it is designed for a monophonic harmonic sound and cannot handle mixtures of harmonic sounds. In this paper, we propose a model (DDSP mixture model) that represents a mixture as the sum of the outputs of multiple pretrained DDSP autoencoders. By fitting the output of the proposed model to the observed mixture, we can directly estimate the synthesis parameters of each source. Through synthesis parameter extraction experiments, we show that the proposed method has high and stable performance compared with a straightforward method that applies the DDSP autoencoder to the signals separated by an audio source separation method.


page 1

page 2

page 3

page 4


Topology in Sound Synthesis and Digital Signal Processing – DAFx2022 Lecture Notes

Lecture notes of a tutorial on topology in sound synthesis and digital s...

DDX7: Differentiable FM Synthesis of Musical Instrument Sounds

FM Synthesis is a well-known algorithm used to generate complex timbre f...

Phase-aware Harmonic/Percussive Source Separation via Convex Optimization

Decomposition of an audio mixture into harmonic and percussive component...

A Generalised Directional Laplacian Distribution: Estimation, Mixture Models and Audio Source Separation

Directional or Circular statistics are pertaining to the analysis and in...

Neural Percussive Synthesis Parameterised by High-Level Timbral Features

We present a deep neural network-based methodology for synthesising perc...

DDSP-based Singing Vocoders: A New Subtractive-based Synthesizer and A Comprehensive Evaluation

A vocoder is a conditional audio generation model that converts acoustic...

Deep Transform: Cocktail Party Source Separation via Probabilistic Re-Synthesis

In cocktail party listening scenarios, the human brain is able to separa...