DeepAI AI Chat
Log In Sign Up

Speech Recognition and Multi-Speaker Diarization of Long Conversations

by   Huanru Henry Mao, et al.
University of California, San Diego

Speech recognition (ASR) and speaker diarization (SD) models have traditionally been trained separately to produce rich conversation transcripts with speaker labels. Recent advances have shown that joint ASR and SD models can learn to leverage audio-lexical inter-dependencies to improve word diarization performance. We introduce a new benchmark of hour-long podcasts collected from the weekly This American Life radio program to better compare these approaches when applied to extended multi-speaker conversations. We find that training separate ASR and SD models perform better when utterance boundaries are known but otherwise joint models can perform better. To handle long conversations with unknown utterance boundaries, we introduce a striding attention decoding algorithm and data augmentation techniques which, combined with model pre-training, improves ASR and SD.


page 1

page 2

page 3

page 4


Joint Speech Recognition and Speaker Diarization via Sequence Transduction

Speech applications dealing with conversations require not only recogniz...

Hypothesis Stitcher for End-to-End Speaker-attributed ASR on Long-form Multi-talker Recordings

An end-to-end (E2E) speaker-attributed automatic speech recognition (SA-...

Understanding Medical Conversations: Rich Transcription, Confidence Scores Information Extraction

In this paper, we describe novel components for extracting clinically re...

A Comparative Study of Modular and Joint Approaches for Speaker-Attributed ASR on Monaural Long-Form Audio

Speaker-attributed automatic speech recognition (SA-ASR) is a task to re...

Towards an Automated SOAP Note: Classifying Utterances from Medical Conversations

Summaries generated from medical conversations can improve recall and un...

PriMock57: A Dataset Of Primary Care Mock Consultations

Recent advances in Automatic Speech Recognition (ASR) have made it possi...

Multiple topic identification in human/human conversations

The paper deals with the automatic analysis of real-life telephone conve...