Accurate and Reliable Confidence Estimation Based on Non-Autoregressive End-to-End Speech Recognition System

05/18/2023

∙

Estimating confidence scores for recognition results is a classic task in ASR field and of vital importance for kinds of downstream tasks and training strategies. Previous end-to-end (E2E) based confidence estimation models (CEM) predict score sequences of equal length with input transcriptions, leading to unreliable estimation when deletion and insertion errors occur. In this paper we proposed CIF-Aligned confidence estimation model (CA-CEM) to achieve accurate and reliable confidence estimation based on novel non-autoregressive E2E ASR model - Paraformer. CA-CEM utilizes the modeling character of continuous integrate-and-fire (CIF) mechanism to generate token-synchronous acoustic embedding, which solves the estimation failure issue above. We measure the quality of estimation with AUC and RMSE in token level and ECE-U - a proposed metrics in utterance level. CA-CEM gains 24 reduction on ECE-U and also better AUC and RMSE on two test sets. Furthermore, we conduct analysis to explore the potential of CEM for different ASR related usage.

READ FULL TEXT

Accurate and Reliable Confidence Estimation Based on Non-Autoregressive End-to-End Speech Recognition System

Multi-Task Learning for End-to-End ASR Word and Utterance Confidence with Deletion Prediction

Utterance-level neural confidence measure for end-to-end children speech recognition

Improving Confidence Estimation on Out-of-Domain Data for End-to-End Speech Recognition

Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model

Learning Word-Level Confidence For Subword End-to-End ASR

Confidence Estimation and Deletion Prediction Using Bidirectional Recurrent Neural Networks

Residual Energy-Based Models for End-to-End Speech Recognition

Accurate and Reliable Confidence Estimation Based on Non-Autoregressive End-to-End Speech Recognition System

Related Research

Multi-Task Learning for End-to-End ASR Word and Utterance Confidence with Deletion Prediction

Utterance-level neural confidence measure for end-to-end children speech recognition

Improving Confidence Estimation on Out-of-Domain Data for End-to-End Speech Recognition

Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model

Learning Word-Level Confidence For Subword End-to-End ASR

Confidence Estimation and Deletion Prediction Using Bidirectional Recurrent Neural Networks

Residual Energy-Based Models for End-to-End Speech Recognition