Deep Learning Based Two-dimensional Speaker Localization With Large Ad-hoc Microphone Arrays

10/19/2022
by   Shupei Liu, et al.
0

Deep learning based speaker localization has shown its advantage in reverberant scenarios. However, it mostly focuses on the direction-of-arrival (DOA) estimation subtask of speaker localization, where the DOA instead of the 2-dimensional (2D) coordinates is obtained only. To obtain the 2D coordinates of multiple speakers with random positions, this paper proposes a deep-learning-based 2D speaker localization method with large ad-hoc microphone arrays, where an ad-hoc microphone array is a set of randomly-distributed microphone nodes with each node set to a traditional microphone array, e.g. a linear array. Specifically, a convolutional neural network is applied to each node to get the direction-of-arrival (DOA) estimation of speech sources. Then, a triangulation and clustering method integrates the DOA estimations of the nodes for estimating the 2D positions of the speech sources. To further improve the estimation accuracy, we propose a softmax-based node selection algorithm. Experimental results with large-scale ad-hoc microphone arrays show that the proposed method achieves significantly better performance than conventional methods in both simulated and real-world environments. The softmax-based node selection further improves the performance.

READ FULL TEXT
research
10/16/2022

End-to-end Two-dimensional Sound Source Localization With Ad-hoc Microphone Arrays

Conventional sound source localization methods are mostly based on a sin...
research
12/01/2020

Deep Ad-hoc Beamforming Based on Speaker Extraction for Target-Dependent Speech Separation

Recently, the research on ad-hoc microphone arrays with deep learning ha...
research
03/29/2021

Scaling sparsemax based channel selection for speech recognition with ad-hoc microphone arrays

Recently, speech recognition with ad-hoc microphone arrays has received ...
research
07/03/2023

Spatial-temporal Graph Based Multi-channel Speaker Verification With Ad-hoc Microphone Arrays

The performance of speaker verification degrades significantly in advers...
research
04/27/2021

One Backward from Ten Forward, Subsampling for Large-Scale Deep Learning

Deep learning models in large-scale machine learning systems are often c...
research
07/01/2021

Attention-based multi-channel speaker verification with ad-hoc microphone arrays

Recently, ad-hoc microphone array has been widely studied. Unlike tradit...
research
06/11/2022

Signal-informed DNN-based DOA Estimation combining an External Microphone and GCC-PHAT Features

Aiming at estimating the direction of arrival (DOA) of a desired speaker...

Please sign up or login with your details

Forgot password? Click here to reset