Monolingual Recognizers Fusion for Code-switching Speech Recognition

11/02/2022
by   Tongtong Song, et al.
0

The bi-encoder structure has been intensively investigated in code-switching (CS) automatic speech recognition (ASR). However, most existing methods require the structures of two monolingual ASR models (MAMs) should be the same and only use the encoder of MAMs. This leads to the problem that pre-trained MAMs cannot be timely and fully used for CS ASR. In this paper, we propose a monolingual recognizers fusion method for CS ASR. It has two stages: the speech awareness (SA) stage and the language fusion (LF) stage. In the SA stage, acoustic features are mapped to two language-specific predictions by two independent MAMs. To keep the MAMs focused on their own language, we further extend the language-aware training strategy for the MAMs. In the LF stage, the BELM fuses two language-specific predictions to get the final prediction. Moreover, we propose a text simulation strategy to simplify the training process of the BELM and reduce reliance on CS data. Experiments on a Mandarin-English corpus show the efficiency of the proposed method. The mix error rate is significantly reduced on the test set after using open-source pre-trained MAMs.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/28/2018

Code-Switching Detection with Data-Augmented Acoustic and Language Models

In this paper, we investigate the code-switching detection performance o...
research
06/18/2020

Multi-Encoder-Decoder Transformer for Code-Switching Speech Recognition

Code-switching (CS) occurs when a speaker alternates words of two or mor...
research
06/29/2022

Language-specific Characteristic Assistance for Code-switching Speech Recognition

Dual-encoder structure successfully utilizes two language-specific encod...
research
06/18/2019

Multi-Graph Decoding for Code-Switching ASR

In the FAME! Project, a code-switching (CS) automatic speech recognition...
research
11/02/2022

Towards Zero-Shot Code-Switched Speech Recognition

In this work, we seek to build effective code-switched (CS) automatic sp...
research
08/11/2023

Bilingual Streaming ASR with Grapheme units and Auxiliary Monolingual Loss

We introduce a bilingual solution to support English as secondary locale...
research
05/01/2020

Style Variation as a Vantage Point for Code-Switching

Code-Switching (CS) is a common phenomenon observed in several bilingual...

Please sign up or login with your details

Forgot password? Click here to reset