Conformer: Convolution-augmented Transformer for Speech Recognition

by   Anmol Gulati, et al.

Recently Transformer and Convolution neural network (CNN) based models have shown promising results in Automatic Speech Recognition (ASR), outperforming Recurrent neural networks (RNNs). Transformer models are good at capturing content-based global interactions, while CNNs exploit local features effectively. In this work, we achieve the best of both worlds by studying how to combine convolution neural networks and transformers to model both local and global dependencies of an audio sequence in a parameter-efficient way. To this regard, we propose the convolution-augmented transformer for speech recognition, named Conformer. Conformer significantly outperforms the previous Transformer and CNN based models achieving state-of-the-art accuracies. On the widely used LibriSpeech benchmark, our model achieves WER of 2.1 using a language model and 1.9 test/testother. We also observe competitive performance of 2.7 small model of only 10M parameters.


page 1

page 2

page 3

page 4


Improving Mandarin Speech Recogntion with Block-augmented Transformer

Recently Convolution-augmented Transformer (Conformer) has shown promisi...

AdaMCT: Adaptive Mixture of CNN-Transformer for Sequential Recommendation

Sequential recommendation (SR) aims to model users' dynamic preferences ...

Conformer LLMs – Convolution Augmented Large Language Models

This work builds together two popular blocks of neural architecture, nam...

Augmenting conformers with structured state space models for online speech recognition

Online speech recognition, where the model only accesses context to the ...

Multi-Head State Space Model for Speech Recognition

State space models (SSMs) have recently shown promising results on small...

A Study of All-Convolutional Encoders for Connectionist Temporal Classification

Connectionist temporal classification (CTC) is a popular sequence predic...

ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context

Convolutional neural networks (CNN) have shown promising results for end...

Please sign up or login with your details

Forgot password? Click here to reset