research
∙
05/24/2023
LMs with a Voice: Spoken Language Modeling beyond Speech Tokens
We present SPECTRON, a novel approach to adapting pre-trained language m...
share
research
∙
05/22/2023
Zero-Shot End-to-End Spoken Language Understanding via Cross-Modal Selective Self-Training
End-to-end (E2E) spoken language understanding (SLU) is constrained by t...
research
∙
07/07/2022
Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation
Large pretrained language models (PLMs) are often domain- or task-adapte...
research
∙
10/24/2020
Align-Refine: Non-Autoregressive Speech Recognition via Iterative Realignment
Non-autoregressive models greatly improve decoding speed over typical se...
research
∙
10/15/2020
Unsupervised Bitext Mining and Translation via Self-trained Contextual Embeddings
We describe an unsupervised method to create pseudo-parallel corpora for...
research
∙
04/30/2020
On the Evaluation of Contextual Embeddings for Zero-Shot Cross-Lingual Transfer Learning
Pre-trained multilingual contextual embeddings have demonstrated state-o...
research
∙
02/12/2020
Attentional Speech Recognition Models Misbehave on Out-of-domain Utterances
We discuss the problem of echographic transcription in autoregressive se...
research
∙
12/03/2019
Deep Contextualized Acoustic Representations For Semi-Supervised Speech Recognition
We propose a novel approach to semi-supervised automatic speech recognit...
research
∙
10/31/2019
Pseudolikelihood Reranking with Masked Language Models
We rerank with scores from pretrained masked language models like BERT t...
research
∙
10/14/2019
Transformers without Tears: Improving the Normalization of Self-Attention
We evaluate three simple, normalization-centric changes to improve Trans...
research
∙
06/30/2019
Contextual Phonetic Pretraining for End-to-end Utterance-level Language and Speaker Recognition
Pretrained contextual word representations in NLP have greatly improved ...
research
∙
01/22/2019