DeepAI AI Chat
Log In Sign Up

Perceive and predict: self-supervised speech representation based loss functions for speech enhancement

by   George Close, et al.

Recent work in the domain of speech enhancement has explored the use of self-supervised speech representations to aid in the training of neural speech enhancement models. However, much of this work focuses on using the deepest or final outputs of self supervised speech representation models, rather than the earlier feature encodings. The use of self supervised representations in such a way is often not fully motivated. In this work it is shown that the distance between the feature encodings of clean and noisy speech correlate strongly with psychoacoustically motivated measures of speech quality and intelligibility, as well as with human Mean Opinion Score (MOS) ratings. Experiments using this distance as a loss function are performed and improved performance over the use of STFT spectrogram distance based loss as well as other common loss functions from speech enhancement literature is demonstrated using objective measures such as perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI).


Self-supervised Learning for Speech Enhancement

Supervised learning for single-channel speech enhancement requires caref...

On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement

Many deep learning-based speech enhancement algorithms are designed to m...

Exploring WavLM on Speech Enhancement

There is a surge in interest in self-supervised learning approaches for ...

A Novel Speech Intelligibility Enhancement Model based on CanonicalCorrelation and Deep Learning

Current deep learning (DL) based approaches to speech intelligibility en...

Self-supervised Learning of Visual Speech Features with Audiovisual Speech Enhancement

We present an introspection of an audiovisual speech enhancement model. ...

Self-Supervised Learning for Speech Enhancement through Synthesis

Modern speech enhancement (SE) networks typically implement noise suppre...