Enhancing Speech Recognition Decoding via Layer Aggregation

03/21/2022
by   Tomer Wullach, et al.
0

Recently proposed speech recognition systems are designed to predict using representations generated by their top layers, employing greedy decoding which isolates each timestep from the rest of the sequence. Aiming for improved performance, a beam search algorithm is frequently utilized and a language model is incorporated to assist with ranking the top candidates. In this work, we experiment with several speech recognition models and find that logits predicted using the top layers may hamper beam search from achieving optimal results. Specifically, we show that fined-tuned Wav2Vec 2.0 and HuBERT yield highly confident predictions, and hypothesize that the predictions are based on local information and may not take full advantage of the information encoded in intermediate layers. To this end, we perform a layer analysis to reveal and visualize how predictions evolve throughout the inference flow. We then propose a prediction method that aggregates the top M layers, potentially leveraging useful information encoded in intermediate layers and relaxing model confidence. We showcase the effectiveness of our approach via beam search decoding, conducting our experiments on Librispeech test and dev sets and achieving WER, and CER reduction of up to 10

READ FULL TEXT

page 2

page 3

research
12/27/2022

Don't Be So Sure! Boosting ASR Decoding via Confidence Relaxation

Automatic Speech Recognition (ASR) systems frequently use a search-based...
research
10/31/2022

Blank Collapse: Compressing CTC emission for the faster decoding

Connectionist Temporal Classification (CTC) model is a very efficient me...
research
05/02/2023

The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers

Applying language models to natural language processing tasks typically ...
research
10/07/2021

Back from the future: bidirectional CTC decoding using future information in speech recognition

In this paper, we propose a simple but effective method to decode the ou...
research
07/20/2023

Transsion TSUP's speech recognition system for ASRU 2023 MADASR Challenge

This paper presents a speech recognition system developed by the Transsi...
research
06/29/2023

Leveraging Cross-Utterance Context For ASR Decoding

While external language models (LMs) are often incorporated into the dec...
research
11/12/2018

Vectorization of hypotheses and speech for faster beam search in encoder decoder-based speech recognition

Attention-based encoder decoder network uses a left-to-right beam search...

Please sign up or login with your details

Forgot password? Click here to reset