Efficient Use of Large Pre-Trained Models for Low Resource ASR

10/26/2022
by   Peter Vieting, et al.
0

Automatic speech recognition (ASR) has been established as a well-performing technique for many scenarios where lots of labeled data is available. Additionally, unsupervised representation learning recently helped to tackle tasks with limited data. Following this, hardware limitations and applications give rise to the question how to efficiently take advantage of large pretrained models and reduce their complexity for downstream tasks. In this work, we study a challenging low resource conversational telephony speech corpus from the medical domain in Vietnamese and German. We show the benefits of using unsupervised techniques beyond simple fine-tuning of large pre-trained models, discuss how to adapt them to a practical telephony task including bandwidth transfer and investigate different data conditions for pre-training and fine-tuning. We outperform the project baselines by 22 pretraining techniques. Further gains of 29 architecture and training and 6

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/17/2021

Efficiently Fusing Pretrained Acoustic and Linguistic Encoders for Low-resource Speech Recognition

End-to-end models have achieved impressive results on the task of automa...
research
08/24/2023

A Small and Fast BERT for Chinese Medical Punctuation Restoration

In clinical dictation, utterances after automatic speech recognition (AS...
research
02/25/2020

Towards Learning a Universal Non-Semantic Representation of Speech

The ultimate goal of transfer learning is to reduce labeled data require...
research
06/09/2023

Developing Speech Processing Pipelines for Police Accountability

Police body-worn cameras have the potential to improve accountability an...
research
04/20/2023

Spaiche: Extending State-of-the-Art ASR Models to Swiss German Dialects

Recent breakthroughs in NLP largely increased the presence of ASR system...
research
02/22/2023

On the contribution of pre-trained models to accuracy and utility in modeling distributed energy resources

Despite their growing popularity, data-driven models of real-world dynam...
research
06/17/2019

Exploiting Unsupervised Pre-training and Automated Feature Engineering for Low-resource Hate Speech Detection in Polish

This paper presents our contribution to PolEval 2019 Task 6: Hate speech...

Please sign up or login with your details

Forgot password? Click here to reset