DeepAI AI Chat
Log In Sign Up

Deep speech inpainting of time-frequency masks

by   Mikolaj Kegler, et al.

In particularly noisy environments, transient loud intrusions can completely overpower parts of the speech signal, leading to an inevitable loss of information. Recent algorithms for noise suppression often yield impressive results but tend to struggle when the signal-to-noise ratio (SNR) of the mixture is low or when parts of the signal are missing. To address these issues, here we introduce an end-to-end framework for the retrieval of missing or severely distorted parts of time-frequency representation of speech, from the short-term context, thus speech inpainting. The framework is based on a convolutional U-Net trained via deep feature losses, obtained through speechVGG, a deep speech feature extractor pre-trained on the word classification task. Our evaluation results demonstrate that the proposed framework is effective at recovering large portions of missing or distorted parts of speech. Specifically, it yields notable improvements in STOI PESQ objective metrics, as assessed using the LibriSpeech dataset.


Speech-VGG: A deep feature extractor for speech processing

A growing number of studies in the field of speech processing employ fea...

Deep Attention Fusion Feature for Speech Separation with End-to-End Post-filter Method

In this paper, we propose an end-to-end post-filter method with deep att...

Speaker Adaptation for End-To-End Speech Recognition Systems in Noisy Environments

We analyze the impact of speaker adaptation in end-to-end architectures ...

A Dual-Staged Context Aggregation Method Towards Efficient End-To-End Speech Enhancement

In speech enhancement, an end-to-end deep neural network converts a nois...

Supervised Initialization of LSTM Networks for Fundamental Frequency Detection in Noisy Speech Signals

Fundamental frequency is one of the most important parameters of human s...

A Multi-Resolution Front-End for End-to-End Speech Anti-Spoofing

The choice of an optimal time-frequency resolution is usually a difficul...