Retrieving Speaker Information from Personalized Acoustic Models for Speech Recognition

11/07/2021
by   Salima Mdhaffar, et al.
0

The widespread of powerful personal devices capable of collecting voice of their users has opened the opportunity to build speaker adapted speech recognition system (ASR) or to participate to collaborative learning of ASR. In both cases, personalized acoustic models (AM), i.e. fine-tuned AM with specific speaker data, can be built. A question that naturally arises is whether the dissemination of personalized acoustic models can leak personal information. In this paper, we show that it is possible to retrieve the gender of the speaker, but also his identity, by just exploiting the weight matrix changes of a neural acoustic model locally adapted to this speaker. Incidentally we observe phenomena that may be useful towards explainability of deep neural networks in the context of speech processing. Gender can be identified almost surely using only the first layers and speaker verification performs well when using middle-up layers. Our experimental study on the TED-LIUM 3 dataset with HMM/TDNN models shows an accuracy of 95 Error Rate of 9.07 weights from personalized models that could be exchanged instead of user data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/06/2021

Privacy attacks for automatic speech recognition acoustic models in a federated learning framework

This paper investigates methods to effectively retrieve speaker informat...
research
07/02/2021

Multi-user VoiceFilter-Lite via Attentive Speaker Embedding

In this paper, we propose a solution to allow speaker conditioned speech...
research
10/08/2020

Gender domain adaptation for automatic speech recognition task

This paper is focused on the finetuning of acoustic models for speaker a...
research
05/16/2019

Articulatory and bottleneck features for speaker-independent ASR of dysarthric speech

The rapid population aging has stimulated the development of assistive d...
research
10/31/2022

An analysis of degenerating speech due to progressive dysarthria on ASR performance

Although personalized automatic speech recognition (ASR) models have rec...
research
04/08/2022

Personal VAD 2.0: Optimizing Personal Voice Activity Detection for On-Device Speech Recognition

Personalization of on-device speech recognition (ASR) has seen explosive...
research
03/14/2019

Audiovisual Speaker Tracking using Nonlinear Dynamical Systems with Dynamic Stream Weights

Data fusion plays an important role in many technical applications that ...

Please sign up or login with your details

Forgot password? Click here to reset