STFT spectral loss for training a neural speech waveform model

10/29/2018

∙

This paper proposes a new loss using short-time Fourier transform (STFT) spectra for the aim of training a high-performance neural speech waveform model that predicts raw continuous speech waveform samples directly. Not only amplitude spectra but also phase spectra obtained from generated speech waveforms are used to calculate the proposed loss. We also mathematically show that training of the waveform model on the basis of the proposed loss can be interpreted as maximum likelihood training that assumes the amplitude and phase spectra of generated speech waveforms following Gaussian and von Mises distributions, respectively. Furthermore, this paper presents a simple network architecture as the speech waveform model, which is composed of uni-directional long short-term memories (LSTMs) and an auto-regressive structure. Experimental results showed that the proposed neural model synthesized high-quality speech waveforms.

READ FULL TEXT

STFT spectral loss for training a neural speech waveform model

Training a Neural Speech Waveform Model using Spectral Losses of Short-Time Fourier Transform and Continuous Wavelet Transform

Long-frame-shift Neural Speech Phase Prediction with Spectral Continuity Enhancement and Interpolation Error Compensation

Maximum Voiced Frequency Estimation: Exploiting Amplitude and Phase Spectra

Electron energy loss spectroscopy database synthesis and automation of core-loss edge recognition by deep-learning neural networks

Discovery of temporal structure intricacy in arterial blood pressure waveforms representing acuity of liver transplant and forecasting short term surgical outcome via unsupervi

Waveform Modeling and Generation Using Hierarchical Recurrent Neural Networks for Speech Bandwidth Extension

A Neural Vocoder Based Packet Loss Concealment Algorithm

STFT spectral loss for training a neural speech waveform model

Related Research

Training a Neural Speech Waveform Model using Spectral Losses of Short-Time Fourier Transform and Continuous Wavelet Transform

Long-frame-shift Neural Speech Phase Prediction with Spectral Continuity Enhancement and Interpolation Error Compensation

Maximum Voiced Frequency Estimation: Exploiting Amplitude and Phase Spectra

Electron energy loss spectroscopy database synthesis and automation of core-loss edge recognition by deep-learning neural networks

Discovery of temporal structure intricacy in arterial blood pressure waveforms representing acuity of liver transplant and forecasting short term surgical outcome via unsupervi

Waveform Modeling and Generation Using Hierarchical Recurrent Neural Networks for Speech Bandwidth Extension

A Neural Vocoder Based Packet Loss Concealment Algorithm