Mandarin-English Code-switching Speech Recognition with Self-supervised Speech Representation Models

10/07/2021
by   Liang-Hsuan Tseng, et al.
0

Code-switching (CS) is common in daily conversations where more than one language is used within a sentence. The difficulties of CS speech recognition lie in alternating languages and the lack of transcribed data. Therefore, this paper uses the recently successful self-supervised learning (SSL) methods to leverage many unlabeled speech data without CS. We show that hidden representations of SSL models offer frame-level language identity even if the models are trained with English speech only. Jointly training CTC and language identification modules with self-supervised speech representations improves CS speech recognition performance. Furthermore, using multilingual speech data for pre-training obtains the best CS speech recognition.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/28/2018

Building a Unified Code-Switching ASR System for South African Languages

We present our first efforts towards building a single multilingual auto...
research
11/01/2018

On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition

Code-switching (CS) refers to a linguistic phenomenon where a speaker us...
research
10/17/2022

Language-agnostic Code-Switching in End-To-End Speech Recognition

Code-Switching (CS) is referred to the phenomenon of alternately using w...
research
04/06/2021

Non-autoregressive Mandarin-English Code-switching Speech Recognition with Pinyin Mask-CTC and Word Embedding Regularization

Mandarin-English code-switching (CS) is frequently used among East and S...
research
11/16/2022

L2 proficiency assessment using self-supervised speech representations

There has been a growing demand for automated spoken language assessment...
research
02/19/2020

Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognition

Recently, language identity information has been utilized to improve the...
research
05/01/2020

Style Variation as a Vantage Point for Code-Switching

Code-Switching (CS) is a common phenomenon observed in several bilingual...

Please sign up or login with your details

Forgot password? Click here to reset