MelGAN-VC: Voice Conversion and Audio Style Transfer on arbitrarily long samples using Spectrograms

by   Marco Pasini, et al.

Traditional voice conversion methods rely on parallel recordings of multiple speakers pronouncing the same sentences. For real-world applications however, parallel data is rarely available. We propose MelGAN-VC, a voice conversion method that relies on non-parallel speech data and is able to convert audio signals of arbitrary length from a source voice to a target voice. We firstly compute spectrograms from waveform data and then perform a domain translation using a Generative Adversarial Network (GAN) architecture. An additional siamese network helps preserving speech information in the translation process, without sacrificing the ability to flexibly model the style of the target speaker. We test our framework with a dataset of clean speech recordings, as well as with a collection of noisy real-world speech examples. Finally, we apply the same method to perform music style transfer, translating arbitrarily long music samples from one genre to another, and showing that our framework is flexible and can be used for audio manipulation applications different from voice conversion.


page 2

page 4

page 5


StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion

We present an unsupervised non-parallel many-to-many voice conversion (V...

Cross-modal Face- and Voice-style Transfer

Image-to-image translation and voice conversion enable the generation of...

On Using Backpropagation for Speech Texture Generation and Voice Conversion

Inspired by recent work on neural network image generation which rely on...

Change your singer: a transfer learning generative adversarial framework for song to song conversion

Have you ever wondered how a song might sound if performed by a differen...

Affective social anthropomorphic intelligent system

Human conversational styles are measured by the sense of humor, personal...

Leveraging Symmetrical Convolutional Transformer Networks for Speech to Singing Voice Style Transfer

In this paper, we propose a model to perform style transfer of speech to...

Improving grapheme-to-phoneme conversion by learning pronunciations from speech recordings

The Grapheme-to-Phoneme (G2P) task aims to convert orthographic input in...

Please sign up or login with your details

Forgot password? Click here to reset