Transformers in Speech Processing: A Survey

03/21/2023
by   Siddique Latif, et al.
1

The remarkable success of transformers in the field of natural language processing has sparked the interest of the speech-processing community, leading to an exploration of their potential for modeling long-range dependencies within speech sequences. Recently, transformers have gained prominence across various speech-related domains, including automatic speech recognition, speech synthesis, speech translation, speech para-linguistics, speech enhancement, spoken dialogue systems, and numerous multimodal applications. In this paper, we present a comprehensive survey that aims to bridge research studies from diverse subfields within speech technology. By consolidating findings from across the speech technology landscape, we provide a valuable resource for researchers interested in harnessing the power of transformers to advance the field. We identify the challenges encountered by transformers in speech processing while also offering insights into potential solutions to address these issues.

READ FULL TEXT
research
10/29/2022

XNOR-FORMER: Learning Accurate Approximations in Long Speech Transformers

Transformers are among the state of the art for many tasks in speech, vi...
research
07/02/2023

Conformer LLMs – Convolution Augmented Large Language Models

This work builds together two popular blocks of neural architecture, nam...
research
06/11/2023

A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks

Transformer is a deep neural network that employs a self-attention mecha...
research
02/15/2022

Transformers in Time Series: A Survey

Transformers have achieved superior performances in many tasks in natura...
research
08/29/2023

Adapting Text-based Dialogue State Tracker for Spoken Dialogues

Although there have been remarkable advances in dialogue systems through...
research
07/17/2023

ivrit.ai: A Comprehensive Dataset of Hebrew Speech for AI Research and Development

We introduce "ivrit.ai", a comprehensive Hebrew speech dataset, addressi...
research
02/01/2023

User Study for Improving Tools for Bible Translation

Technology has increasingly become an integral part of the Bible transla...

Please sign up or login with your details

Forgot password? Click here to reset