DeepAI AI Chat
Log In Sign Up

Advances in Online Audio-Visual Meeting Transcription

by   Takuya Yoshioka, et al.

This paper describes a system that generates speaker-annotated transcripts of meetings by using a microphone array and a 360-degree camera. The hallmark of the system is its ability to handle overlapped speech, which has been an unsolved problem in realistic settings for over a decade. We show that this problem can be addressed by using a continuous speech separation approach. In addition, we describe an online audio-visual speaker diarization method that leverages face tracking and identification, sound source localization, speaker identification, and, if available, prior speaker information for robustness to various real world challenges. All components are integrated in a meeting transcription framework called SRD, which stands for "separate, recognize, and diarize". Experimental results using recordings of natural meetings involving up to 11 attendees are reported. The continuous speech separation improves a word error rate (WER) by 16.1 complete list of meeting attendees is available, the discrepancy between WER and speaker-attributed WER is only 1.0 association. This increases marginally to 1.6 unknown to the system.


page 1

page 2

page 3

page 4


Meeting Transcription Using Virtual Microphone Arrays

We describe a system that generates speaker-annotated transcripts of mee...

A Real-Time Active Speaker Detection System Integrating an Audio-Visual Signal with a Spatial Querying Mechanism

We introduce a distinctive real-time, causal, neural network-based activ...

Advanced Rich Transcription System for Estonian Speech

This paper describes the current TTÜ speech transcription system for Est...

LOCATA challenge: speaker localization with a planar array

This document describes our submission to the 2018 LOCalization And TrAc...

Blind Speech Separation and Dereverberation using Neural Beamforming

In this paper, we present the Blind Speech Separation and Dereverberatio...

Microsoft Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2020

This paper describes the Microsoft speaker diarization system for monaur...

Diarization of Legal Proceedings. Identifying and Transcribing Judicial Speech from Recorded Court Audio

United States Courts make audio recordings of oral arguments available a...