Adaptation of Whisper models to child speech recognition

by   Rishabh Jain, et al.

Automatic Speech Recognition (ASR) systems often struggle with transcribing child speech due to the lack of large child speech datasets required to accurately train child-friendly ASR models. However, there are huge amounts of annotated adult speech datasets which were used to create multilingual ASR models, such as Whisper. Our work aims to explore whether such models can be adapted to child speech to improve ASR for children. In addition, we compare Whisper child-adaptations with finetuned self-supervised models, such as wav2vec2. We demonstrate that finetuning Whisper on child speech yields significant improvements in ASR performance on child speech, compared to non finetuned Whisper models. Additionally, utilizing self-supervised Wav2vec2 models that have been finetuned on child speech outperforms Whisper finetuning.


page 1

page 2

page 3

page 4


Can Self-Supervised Learning solve the problem of child speech recognition?

Despite recent advancements in deep learning technologies, Child Speech ...

Improving Accented Speech Recognition with Multi-Domain Training

Thanks to the rise of self-supervised learning, automatic speech recogni...

EAT: Enhanced ASR-TTS for Self-supervised Speech Recognition

Self-supervised ASR-TTS models suffer in out-of-domain data conditions. ...

Towards Representative Subset Selection for Self-Supervised Speech Recognition

Self-supervised speech recognition models require considerable labeled t...

Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separator

Multi-talker overlapped speech poses a significant challenge for speech ...

BEA-Base: A Benchmark for ASR of Spontaneous Hungarian

Hungarian is spoken by 15 million people, still, easily accessible Autom...

Avoid Overthinking in Self-Supervised Models for Speech Recognition

Self-supervised learning (SSL) models reshaped our approach to speech, l...

Please sign up or login with your details

Forgot password? Click here to reset