Lattention: Lattice-attention in ASR rescoring

11/19/2021
by   Prabhat Pandey, et al.
3

Lattices form a compact representation of multiple hypotheses generated from an automatic speech recognition system and have been shown to improve performance of downstream tasks like spoken language understanding and speech translation, compared to using one-best hypothesis. In this work, we look into the effectiveness of lattice cues for rescoring n-best lists in second-pass. We encode lattices with a recurrent network and train an attention encoder-decoder model for n-best rescoring. The rescoring model with attention to lattices achieves 4-5 attention to both lattices and acoustic features. We show that rescoring models with attention to lattices outperform models with attention to n-best hypotheses. We also study different ways to incorporate lattice weights in the lattice encoder and demonstrate their importance for n-best rescoring.

READ FULL TEXT
research
11/02/2020

Adapting Pretrained Transformer to Lattices for Spoken Language Understanding

Lattices are compact representations that encode multiple hypotheses, su...
research
03/08/2021

A Parallelizable Lattice Rescoring Strategy with Neural Language Models

This paper proposes a parallel computation strategy and a posterior-base...
research
08/21/2000

Processing Self Corrections in a speech to speech system

Speech repairs occur often in spontaneous spoken dialogues. The ability ...
research
06/11/2021

N-Best ASR Transformer: Enhancing SLU Performance using Multiple ASR Hypotheses

Spoken Language Understanding (SLU) systems parse speech into semantic s...
research
11/06/2018

Discriminative training of RNNLMs with the average word error criterion

In automatic speech recognition (ASR), recurrent neural language models ...
research
06/04/2019

Self-Attentional Models for Lattice Inputs

Lattices are an efficient and effective method to encode ambiguity of up...
research
04/03/2017

Neural Lattice-to-Sequence Models for Uncertain Inputs

The input to a neural sequence-to-sequence model is often determined by ...

Please sign up or login with your details

Forgot password? Click here to reset