DeepAI AI Chat
Log In Sign Up

VQCPC-GAN: Variable-length Adversarial Audio Synthesis using Vector-Quantized Contrastive Predictive Coding

05/04/2021
by   Javier Nistal, et al.
0

Influenced by the field of Computer Vision, Generative Adversarial Networks (GANs) are often adopted for the audio domain using fixed-size two-dimensional spectrogram representations as the "image data". However, in the (musical) audio domain, it is often desired to generate output of variable duration. This paper presents VQCPC-GAN, an adversarial framework for synthesizing variable-length audio by exploiting Vector-Quantized Contrastive Predictive Coding (VQCPC). A sequence of VQCPC tokens extracted from real audio data serves as conditional input to a GAN architecture, providing step-wise time-dependent features of the generated content. The input noise z (characteristic in adversarial architectures) remains fixed over time, ensuring temporal consistency of global features. We evaluate the proposed model by comparing a diverse set of metrics against various strong baselines. Results show that, even though the baselines score best, VQCPC-GAN achieves comparable performance even when generating variable-length audio. Numerous sound examples are provided in the accompanying website, and we release the code for reproducibility.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/03/2021

DarkGAN: Exploiting Knowledge Distillation for Comprehensible Audio Synthesis with GANs

Generative Adversarial Networks (GANs) have achieved excellent audio syn...
05/18/2020

Unconditional Audio Generation with Generative Adversarial Networks and Cycle Regularization

In a recent paper, we have presented a generative adversarial network (G...
06/16/2020

Comparing Representations for Audio Synthesis Using Generative Adversarial Networks

In this paper, we compare different audio signal representations, includ...
02/11/2019

Adversarial Generation of Time-Frequency Features with application in audio synthesis

Time-frequency (TF) representations provide powerful and intuitive featu...
03/12/2021

Signal Representations for Synthesizing Audio Textures with Generative Adversarial Networks

Generative Adversarial Networks (GANs) currently achieve the state-of-th...
04/05/2018

Feedback GAN (FBGAN) for DNA: a Novel Feedback-Loop Architecture for Optimizing Protein Functions

Generative Adversarial Networks (GANs) represent an attractive and novel...
06/05/2022

Diffusion-GAN: Training GANs with Diffusion

For stable training of generative adversarial networks (GANs), injecting...