b'Wei-Ning Hsu'

research

∙ 08/10/2023

EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis

Recent work has shown that it is possible to resynthesize high-quality s...

0 Tu Anh Nguyen, et al. ∙

research

∙ 06/23/2023

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale

Large-scale generative models such as GPT and DALL-E have revolutionized...

0 Matthew Le, et al. ∙

research

∙ 05/22/2023

Scaling Speech Technology to 1,000+ Languages

Expanding the language coverage of speech technology has the potential t...

0 Vineel Pratap, et al. ∙

research

∙ 05/17/2023

DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning

In this paper, we introduce self-distillation and online clustering for ...

0 Alexander H. Liu, et al. ∙

research

∙ 03/20/2023

Cocktail HuBERT: Generalized Self-Supervised Pre-training for Mixture and Single-Source Speech

Self-supervised learning leverages unlabeled data effectively, improving...

0 Maryam Fazel-Zarandi, et al. ∙

research

∙ 03/01/2023

MuAViC: A Multilingual Audio-Visual Corpus for Robust Speech Recognition and Robust Speech-to-Text Translation

We introduce MuAViC, a multilingual audio-visual corpus for robust speec...

0 Mohamed Anwar, et al. ∙

research

∙ 02/10/2023

AV-data2vec: Self-supervised Learning of Audio-Visual Speech Representations with Contextualized Target Representations

Self-supervision has shown great potential for audio-visual speech recog...

0 Jiachen Lian, et al. ∙

research

∙ 01/10/2023

Scaling Laws for Generative Mixed-Modal Language Models

Generative language models define distributions over sequences of tokens...

0 Armen Aghajanyan, et al. ∙

research

∙ 12/21/2022

ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement

Prior works on improving speech quality with visual input typically stud...

0 Wei-Ning Hsu, et al. ∙

research

∙ 12/14/2022

Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language

Current self-supervised learning algorithms are often modality-specific ...

0 Alexei Baevski, et al. ∙

research

∙ 12/02/2022

Continual Learning for On-Device Speech Recognition using Disentangled Conformers

Automatic speech recognition research focuses on training and evaluating...

0 Anuj Diwan, et al. ∙

research

∙ 11/11/2022

Speech-to-Speech Translation For A Real-world Unwritten Language

We study speech-to-speech translation (S2ST) that translates speech from...

0 Peng-Jen Chen, et al. ∙

research

∙ 10/18/2022

Simple and Effective Unsupervised Speech Translation

The amount of labeled data to train models for speech tasks is limited f...

0 Changhan Wang, et al. ∙

research

∙ 07/14/2022

A Single Self-Supervised Model for Many Speech Modalities Enables Zero-Shot Modality Transfer

While audio-visual speech models can yield superior performance and robu...

0 Wei-Ning Hsu, et al. ∙

research

∙ 05/15/2022

Learning Lip-Based Audio-Visual Speaker Embeddings with AV-HuBERT

This paper investigates self-supervised pre-training for audio-visual sp...

0 Bowen Shi, et al. ∙

research

∙ 04/25/2022

On-demand compute reduction with stochastic wav2vec 2.0

Squeeze and Efficient Wav2vec (SEW) is a recently proposed architecture ...

0 Apoorv Vyas, et al. ∙

research

∙ 04/11/2022

Unified Speech-Text Pre-training for Speech Translation and Recognition

We describe a method to jointly pre-train speech and text in an encoder-...

1 Yun Tang, et al. ∙

research

∙ 04/06/2022

Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation

Direct speech-to-speech translation (S2ST) models suffer from data scarc...

0 Sravya Popuri, et al. ∙

research

∙ 04/06/2022

Simple and Effective Unsupervised Speech Synthesis

We introduce the first unsupervised speech synthesis system based on a s...

0 Alexander H. Liu, et al. ∙

research

∙ 04/05/2022

Towards End-to-end Unsupervised Speech Recognition

Unsupervised speech recognition has shown great potential to make Automa...

0 Alexander H. Liu, et al. ∙

research

∙ 03/30/2022

Generative Spoken Dialogue Language Modeling

We introduce dGSLM, the first "textless" model able to generate audio sa...

5 Tu Anh Nguyen, et al. ∙

research

∙ 03/01/2022

Measuring the Impact of Individual Domain Factors in Self-Supervised Pre-Training

Human speech data comprises a rich set of domain factors such as accent,...

0 Ramon Sanabria, et al. ∙

research

∙ 02/15/2022

textless-lib: a Library for Textless Spoken Language Processing

Textless spoken language processing research aims to extend the applicab...

11 Eugene Kharitonov, et al. ∙

research

∙ 02/07/2022

data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language

While the general idea of self-supervised learning is identical across m...

0 Alexei Baevski, et al. ∙

research

∙ 01/05/2022

Robust Self-Supervised Audio-Visual Speech Recognition

Audio-based automatic speech recognition (ASR) degrades significantly in...

0 Bowen Shi, et al. ∙

research

∙ 01/05/2022

Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction

Video recordings of speech contain correlated audio and visual informati...

0 Bowen Shi, et al. ∙

research

∙ 12/15/2021

Textless Speech-to-Speech Translation on Real Data

We present a textless speech-to-speech translation (S2ST) system that ca...

0 Ann Lee, et al. ∙

research

∙ 11/14/2021

Textless Speech Emotion Conversion using Decomposed and Discrete Representations

Speech emotion conversion is the task of modifying the perceived emotion...

6 Felix Kreuk, et al. ∙

research

∙ 10/15/2021

Direct simultaneous speech to speech translation

We present the first direct simultaneous speech-to-speech translation (S...

0 Xutai Ma, et al. ∙

research

∙ 09/14/2021

fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkit

This paper presents fairseq S^2, a fairseq extension for speech synthesi...

0 Changhan Wang, et al. ∙

research

∙ 09/07/2021

Text-Free Prosody-Aware Generative Spoken Language Modeling

Speech pre-training has primarily demonstrated efficacy on classificatio...

14 Eugene Kharitonov, et al. ∙

research

∙ 07/12/2021

Direct speech-to-speech translation with discrete units

We present a direct speech-to-speech translation (S2ST) model that trans...

0 Ann Lee, et al. ∙

research

∙ 06/14/2021

Kaizen: Continuously improving teacher using Exponential Moving Average for semi-supervised speech recognition

In this paper, we introduce the Kaizen framework that uses a continuousl...

0 Vimal Manohar, et al. ∙

research

∙ 06/14/2021

HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units

Self-supervised approaches for speech representation learning are challe...

0 Wei-Ning Hsu, et al. ∙

research

∙ 05/24/2021

Unsupervised Speech Recognition

Despite rapid progress in the recent past, current speech recognition sy...

0 Alexei Baevski, et al. ∙

research

∙ 04/02/2021

Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training

Self-supervised learning of speech representations has been a very activ...

0 Wei-Ning Hsu, et al. ∙

research

∙ 04/01/2021

Speech Resynthesis from Discrete Disentangled Self-Supervised Representations

We propose using self-supervised discrete representations for the task o...

10 Adam Polyak, et al. ∙

research

∙ 02/01/2021

Generative Spoken Language Modeling from Raw Audio

Generative spoken language modeling involves learning jointly the acoust...

11 Kushal Lakhotia, et al. ∙

research

∙ 12/31/2020

Text-Free Image-to-Speech Synthesis Using Learned Segmental Units

In this paper we present the first model for directly synthesizing fluen...

12 Wei-Ning Hsu, et al. ∙

research

∙ 10/02/2020

Differentiable Weighted Finite-State Transducers

We introduce a framework for automatic differentiation with weighted fin...

6 Awni Hannun, et al. ∙

research

∙ 06/03/2020

A Convolutional Deep Markov Model for Unsupervised Speech Representation Learning

Probabilistic Latent Variable Models (LVMs) provide an alternative to se...

0 Sameer Khurana, et al. ∙

research

∙ 02/24/2020

Semi-Supervised Speech Recognition via Local Prior Matching

For sequence transduction tasks like speech recognition, a strong struct...

0 Wei-Ning Hsu, et al. ∙

research

∙ 11/21/2019

Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech

In this paper, we present a method for learning discrete linguistic unit...

0 David Harwath, et al. ∙

research

∙ 07/09/2019

Transfer Learning from Audio-Visual Grounding to Speech Recognition

Transfer learning aims to reduce the amount of data required to excel at...

0 Wei-Ning Hsu, et al. ∙

research

∙ 04/05/2019

An Unsupervised Autoregressive Model for Speech Representation Learning

This paper proposes a novel unsupervised autoregressive neural model for...

0 Yu-An Chung, et al. ∙

research

∙ 02/21/2019

Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling

Lingvo is a Tensorflow framework offering a complete solution for collab...

13 Jonathan Shen, et al. ∙

research

∙ 10/16/2018

Hierarchical Generative Modeling for Controllable Speech Synthesis

This paper proposes a neural end-to-end text-to-speech (TTS) model which...

0 Wei-Ning Hsu, et al. ∙

research

∙ 09/12/2018

Unsupervised Representation Learning of Speech for Dialect Identification

In this paper, we explore the use of a factorized hierarchical variation...

0 Suwon Shon, et al. ∙

research

∙ 08/30/2018

Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesis

Although end-to-end text-to-speech (TTS) models such as Tacotron have sh...

0 Yu-An Chung, et al. ∙

research

∙ 06/13/2018

Unsupervised Adaptation with Interpretable Disentangled Representations for Distant Conversational Speech Recognition

The current trend in automatic speech recognition is to leverage large a...

0 Wei-Ning Hsu, et al. ∙

Wei-Ning Hsu

Featured Co-authors