Log In Sign Up

Adversarial Audio Synthesis with Complex-valued Polynomial Networks

by   Yongtao Wu, et al.

Time-frequency (TF) representations in audio synthesis have been increasingly modeled with real-valued networks. However, overlooking the complex-valued nature of TF representations can result in suboptimal performance and require additional modules (e.g., for modeling the phase). To this end, we introduce complex-valued polynomial networks, called APOLLO, that integrate such complex-valued representations in a natural way. Concretely, APOLLO captures high-order correlations of the input elements using high-order tensors as scaling parameters. By leveraging standard tensor decompositions, we derive different architectures and enable modeling richer correlations. We outline such architectures and showcase their performance in audio generation across four benchmarks. As a highlight, APOLLO results in 17.5% improvement over adversarial methods and 8.2% over the state-of-the-art diffusion models on SC09 dataset in audio generation. Our models can encourage the systematic design of other efficient architectures on the complex field.


page 7

page 19

page 27

page 28


Deep Complex Networks

At present, the vast majority of building blocks, techniques, and archit...

Comparing Representations for Audio Synthesis Using Generative Adversarial Networks

In this paper, we compare different audio signal representations, includ...

Adversarial Generation of Time-Frequency Features with application in audio synthesis

Time-frequency (TF) representations provide powerful and intuitive featu...

Co-domain Symmetry for Complex-Valued Deep Learning

We study complex-valued scaling as a type of symmetry natural and unique...

Filterbank design for end-to-end speech separation

Single-channel speech separation has recently made great progress thanks...

Polynomial Networks in Deep Classifiers

Deep neural networks have been the driving force behind the success in c...