DeepAI
Log In Sign Up

Adversarial Audio Synthesis with Complex-valued Polynomial Networks

06/14/2022
by   Yongtao Wu, et al.
0

Time-frequency (TF) representations in audio synthesis have been increasingly modeled with real-valued networks. However, overlooking the complex-valued nature of TF representations can result in suboptimal performance and require additional modules (e.g., for modeling the phase). To this end, we introduce complex-valued polynomial networks, called APOLLO, that integrate such complex-valued representations in a natural way. Concretely, APOLLO captures high-order correlations of the input elements using high-order tensors as scaling parameters. By leveraging standard tensor decompositions, we derive different architectures and enable modeling richer correlations. We outline such architectures and showcase their performance in audio generation across four benchmarks. As a highlight, APOLLO results in 17.5% improvement over adversarial methods and 8.2% over the state-of-the-art diffusion models on SC09 dataset in audio generation. Our models can encourage the systematic design of other efficient architectures on the complex field.

READ FULL TEXT

page 7

page 19

page 27

page 28

05/27/2017

Deep Complex Networks

At present, the vast majority of building blocks, techniques, and archit...
06/16/2020

Comparing Representations for Audio Synthesis Using Generative Adversarial Networks

In this paper, we compare different audio signal representations, includ...
02/11/2019

Adversarial Generation of Time-Frequency Features with application in audio synthesis

Time-frequency (TF) representations provide powerful and intuitive featu...
12/02/2021

Co-domain Symmetry for Complex-Valued Deep Learning

We study complex-valued scaling as a type of symmetry natural and unique...
10/23/2019

Filterbank design for end-to-end speech separation

Single-channel speech separation has recently made great progress thanks...
04/16/2021

Polynomial Networks in Deep Classifiers

Deep neural networks have been the driving force behind the success in c...