Acoustic feature learning using cross-domain articulatory measurements

03/19/2018
by   Qingming Tang, et al.
0

Previous work has shown that it is possible to improve speech recognition by learning acoustic features from paired acoustic-articulatory data, for example by using canonical correlation analysis (CCA) or its deep extensions. One limitation of this prior work is that the learned feature models are difficult to port to new datasets or domains, and articulatory data is not available for most speech corpora. In this work we study the problem of acoustic feature learning in the setting where we have access to an external, domain-mismatched dataset of paired speech and articulatory measurements, either with or without labels. We develop methods for acoustic feature learning in these settings, based on deep variational CCA and extensions that use both source and target domain data and labels. Using this approach, we improve phonetic recognition accuracies on both TIMIT and Wall Street Journal and analyze a number of design choices.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/19/2018

Acoustic feature learning cross-domain articulatory measurements

Previous work has shown that it is possible to improve speech recognitio...
research
08/11/2017

Acoustic Feature Learning via Deep Variational Canonical Correlation Analysis

We study the problem of acoustic feature learning in the setting where w...
research
04/25/2018

End-to-End Multimodal Speech Recognition

Transcription or sub-titling of open-domain videos is still a challengin...
research
06/20/2022

Boosting Cross-Domain Speech Recognition with Self-Supervision

The cross-domain performance of automatic speech recognition (ASR) could...
research
02/13/2016

Signer-independent Fingerspelling Recognition with Deep Neural Network Adaptation

We study the problem of recognition of fingerspelled letter sequences in...
research
04/27/2023

Cross-Domain Evaluation of POS Taggers: From Wall Street Journal to Fandom Wiki

The Wall Street Journal section of the Penn Treebank has been the de-fac...

Please sign up or login with your details

Forgot password? Click here to reset