DeepAI AI Chat
Log In Sign Up

GANSynth: Adversarial Neural Audio Synthesis

02/23/2019
by   Jesse Engel, et al.
20

Efficient audio synthesis is an inherently difficult machine learning task, as human perception is sensitive to both global structure and fine-scale waveform coherence. Autoregressive models, such as WaveNet, model local structure at the expense of global latent structure and slow iterative sampling, while Generative Adversarial Networks (GANs), have global latent conditioning and efficient parallel sampling, but struggle to generate locally-coherent audio waveforms. Herein, we demonstrate that GANs can in fact generate high-fidelity and locally-coherent audio by modeling log magnitudes and instantaneous frequencies with sufficient frequency resolution in the spectral domain. Through extensive empirical investigations on the NSynth dataset, we demonstrate that GANs are able to outperform strong WaveNet baselines on automated and human evaluation metrics, and efficiently generate audio several orders of magnitude faster than their autoregressive counterparts.

READ FULL TEXT

page 2

page 6

page 7

page 8

page 13

page 14

10/12/2020

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

Several recent work on speech synthesis have employed generative adversa...
06/16/2020

Comparing Representations for Audio Synthesis Using Generative Adversarial Networks

In this paper, we compare different audio signal representations, includ...
09/21/2020

DiffWave: A Versatile Diffusion Model for Audio Synthesis

In this work, we propose DiffWave, a versatile Diffusion probabilistic m...
10/08/2019

MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis

Previous works <cit.> have found that generating coherent raw audio wave...
11/06/2018

FloWaveNet : A Generative Flow for Raw Audio

Most of modern text-to-speech architectures use a WaveNet vocoder for sy...
08/03/2021

DarkGAN: Exploiting Knowledge Distillation for Comprehensible Audio Synthesis with GANs

Generative Adversarial Networks (GANs) have achieved excellent audio syn...
06/01/2023

Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis

Recent advancements in neural vocoding are predominantly driven by Gener...

Code Repositories

GANsynth-pytorch

A simplified PyTorch implementation of GANsynth


view repo

ADVERSARIAL-NEURAL-AUDIO-SYNTHESIS

ADVERSARIAL NEURAL AUDIO SYNTHESISを読んでメモ取ってみたもの


view repo